~sircmpwn/sr.ht-discuss

12 3

https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/

Constantine A. Murenin
Details
Message ID
<20190913233613.GS8841@Cns.Cns.SU>
DKIM signature
missing
Download raw message
> Hey! Are you a NetBSD expert? We need some help to finish up NetBSD support on builds.sr.ht. Get in touch?

What help do you need?

C.
Details
Message ID
<BX1KPCV66VDS.3ENFHO2I9IN6K@homura>
In-Reply-To
<20190913233613.GS8841@Cns.Cns.SU> (view parent)
DKIM signature
pass
Download raw message
On Fri Sep 13, 2019 at 4:36 PM Constantine A. Murenin wrote:
> > Hey! Are you a NetBSD expert? We need some help to finish up NetBSD
> > support on builds.sr.ht. Get in touch?
> 
> What help do you need?

Hey Constantine, thanks for writing!

The NetBSD image is almost complete, you can see the genimg script
here:

https://git.sr.ht/~sircmpwn/builds.sr.ht/tree/master/images/netbsd

If you clone this repo on a NetBSD box and pop over to the 8.x or
current directories, then run ./genimg as root, it'll produce a working
NetBSD image (or at least it did when I last touched it).

The problem was that I'm unable to get it to resize itself to fill the
available space on the first boot. Since we use the images to generate
themselves, this means that the NetBSD image will get smaller and
smaller with each successive run and will eventually be too small to
accomodate builds.
Constantine A. Murenin
Details
Message ID
<20190916221904.GX8841@Cns.Cns.SU>
In-Reply-To
<BX1KPCV66VDS.3ENFHO2I9IN6K@homura> (view parent)
DKIM signature
missing
Download raw message
On 2019-W38-1 12:39 -0400, Drew DeVault wrote:
> On Fri Sep 13, 2019 at 4:36 PM Constantine A. Murenin wrote:
> > > Hey! Are you a NetBSD expert? We need some help to finish up NetBSD
> > > support on builds.sr.ht. Get in touch?
> > 
> > What help do you need?
> 
> Hey Constantine, thanks for writing!
> 
> The NetBSD image is almost complete, you can see the genimg script
> here:
> 
> https://git.sr.ht/~sircmpwn/builds.sr.ht/tree/master/images/netbsd
> 
> If you clone this repo on a NetBSD box and pop over to the 8.x or
> current directories, then run ./genimg as root, it'll produce a working
> NetBSD image (or at least it did when I last touched it).
> 
> The problem was that I'm unable to get it to resize itself to fill the
> available space on the first boot. Since we use the images to generate
> themselves, this means that the NetBSD image will get smaller and
> smaller with each successive run and will eventually be too small to
> accomodate builds.

I see you're already using `resize_root=YES` in 
https://git.sr.ht/~sircmpwn/builds.sr.ht/tree/master/images/netbsd/genimg

Does this mean that the http://bxr.su/n/etc/rc.d/resize_root script isn't 
working in your situation?

I actually spoke with Jeff (riz@) about this recently 
in regards to http://wiki.netbsd.org/amazon_ec2/amis/; 
I don't quite understand how resizing works yet, 
because you'd need to make sure that the 
fdisk and disklabel are setup properly before 
http://mdoc.su/n/resize_ffs.8 would work.

C.
Details
Message ID
<BX1U9PD070XZ.7B2FKFZ0VBOA@homura>
In-Reply-To
<20190916221904.GX8841@Cns.Cns.SU> (view parent)
DKIM signature
pass
Download raw message
Yeah, it seems like resize_root isn't doing the trick. I started
debugging it but if I recall correctly, I had a hard time getting logs
out of it.
Constantine A. Murenin
Details
Message ID
<20190929225857.GF8841@Cns.Cns.SU>
In-Reply-To
<BX1U9PD070XZ.7B2FKFZ0VBOA@homura> (view parent)
DKIM signature
missing
Download raw message
On 2019-W38-1 20:08 -0400, Drew DeVault wrote:
> Yeah, it seems like resize_root isn't doing the trick. I started
> debugging it but if I recall correctly, I had a hard time getting logs
> out of it.
> 

The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely 
zero mentions of fdisk or disklabel, so, I don't really understand 
how it's supposed to work without modifying the partition tables.

I've tried running it in VirtualBox after changing the size of the 
underlying disk, and, no surprise, the script doesn't do anything at all.

Is there a way to handle fdisk/disklabel in the best way, or to create 
a bootable i386/amd64 disk image without fdisk/disklabel?

Cc'ing chopps@ and jmcneill@:  do you know how we're supposed to handle 
fdisk/disklabel change here?

C.
Details
Message ID
<BXCXKE6XDXSG.1KJKGI9KGCZ0N@homura>
In-Reply-To
<20190929225857.GF8841@Cns.Cns.SU> (view parent)
DKIM signature
pass
Download raw message
On Sun Sep 29, 2019 at 3:58 PM Constantine A. Murenin wrote:
> The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely 
> zero mentions of fdisk or disklabel, so, I don't really understand 
> how it's supposed to work without modifying the partition tables.

Maybe it's designed to fill the partition? In that case it won't be
helpful for my needs and a different solution will have to be found.
I'm scant on ideas, though, because making a disk as large as the host
system's disk doesn't seem feasible without some way of mounting a qcow2
with nbd or something similar.
Constantine A. Murenin
Details
Message ID
<20190930014103.GG8841@Cns.Cns.SU>
In-Reply-To
<BXCXKE6XDXSG.1KJKGI9KGCZ0N@homura> (view parent)
DKIM signature
missing
Download raw message
On 2019-W39-7 21:03 -0400, Drew DeVault wrote:
> On Sun Sep 29, 2019 at 3:58 PM Constantine A. Murenin wrote:
> > The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely 
> > zero mentions of fdisk or disklabel, so, I don't really understand 
> > how it's supposed to work without modifying the partition tables.
> 
> Maybe it's designed to fill the partition? In that case it won't be
> helpful for my needs and a different solution will have to be found.
> I'm scant on ideas, though, because making a disk as large as the host
> system's disk doesn't seem feasible without some way of mounting a qcow2
> with nbd or something similar.

OK, I think I found how this works, thanks to some mention of the other rc.d:

https://www.irascible.net/blog/netbsd-on-the-raspberry-pi/

resize_disklabel=NO
resize_root=NO

http://bxr.su/n/distrib/utils/embedded/files/resize_disklabel

You can fetch it raw, it seems to work great:

cd /etc/rc.d/
ftp http://bxr.su/raw/NetBSD/distrib/utils/embedded/files/resize_disklabel
chmod +x resize_disklabel

Not sure why it's so well hidden -- resize_root without resize_disklabel 
seems pretty useless.  Why would you even ever call resize_root without 
resize_disklabel?  Resize_disklabel does seem to handle fdisk, too, BTW.

C.
Constantine A. Murenin
Details
Message ID
<20190930022047.GH8841@Cns.Cns.SU>
In-Reply-To
<20190930014103.GG8841@Cns.Cns.SU> (view parent)
DKIM signature
missing
Download raw message
On 2019-W39-7 18:41 -0700, Constantine A. Murenin wrote:
> On 2019-W39-7 21:03 -0400, Drew DeVault wrote:
> > On Sun Sep 29, 2019 at 3:58 PM Constantine A. Murenin wrote:
> > > The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely 
> > > zero mentions of fdisk or disklabel, so, I don't really understand 
> > > how it's supposed to work without modifying the partition tables.
> > 
> > Maybe it's designed to fill the partition? In that case it won't be
> > helpful for my needs and a different solution will have to be found.
> > I'm scant on ideas, though, because making a disk as large as the host
> > system's disk doesn't seem feasible without some way of mounting a qcow2
> > with nbd or something similar.
> 
> OK, I think I found how this works, thanks to some mention of the other rc.d:
> 
> https://www.irascible.net/blog/netbsd-on-the-raspberry-pi/
> 
> resize_disklabel=NO
> resize_root=NO
> 
> http://bxr.su/n/distrib/utils/embedded/files/resize_disklabel
> 
> You can fetch it raw, it seems to work great:
> 
> cd /etc/rc.d/
> ftp http://bxr.su/raw/NetBSD/distrib/utils/embedded/files/resize_disklabel
> chmod +x resize_disklabel
> 
> Not sure why it's so well hidden -- resize_root without resize_disklabel 
> seems pretty useless.  Why would you even ever call resize_root without 
> resize_disklabel?  Resize_disklabel does seem to handle fdisk, too, BTW.

BTW, you probably don't want to run `resize_root onestart` once the system 
is running multiuser -- it passes `-y` to resize_ffs, which causes 
temporary filesystem corruption if the filesystem is running in rw mode.

C.

> 
> C.
> 
Christian Hopps
Details
Message ID
<9C609703-F75E-4729-B804-3A26C621C88E@chopps.org>
In-Reply-To
<20190929225857.GF8841@Cns.Cns.SU> (view parent)
DKIM signature
missing
Download raw message
So I originally coded this to support booting netbsd images (e.g., AMI amazon) in the cloud which would resize their root according to the disk allocated to them. I haven't had a chance to use it in a while and I believe it's undergone changes/improvements.

Thanks,
Chris.

> On Sep 29, 2019, at 6:58 PM, Constantine A. Murenin <cnst@NetBSD.org> wrote:
> 
> On 2019-W38-1 20:08 -0400, Drew DeVault wrote:
>> Yeah, it seems like resize_root isn't doing the trick. I started
>> debugging it but if I recall correctly, I had a hard time getting logs
>> out of it.
>> 
> 
> The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely
> zero mentions of fdisk or disklabel, so, I don't really understand
> how it's supposed to work without modifying the partition tables.
> 
> I've tried running it in VirtualBox after changing the size of the
> underlying disk, and, no surprise, the script doesn't do anything at all.
> 
> Is there a way to handle fdisk/disklabel in the best way, or to create
> a bootable i386/amd64 disk image without fdisk/disklabel?
> 
> Cc'ing chopps@ and jmcneill@:  do you know how we're supposed to handle
> fdisk/disklabel change here?
> 
> C.
> 
Constantine A. Murenin
Details
Message ID
<20191001054257.GI8841@Cns.Cns.SU>
In-Reply-To
<9C609703-F75E-4729-B804-3A26C621C88E@chopps.org> (view parent)
DKIM signature
missing
Download raw message
On 2019-W40-1 20:16 -0400, Christian Hopps wrote:
> So I originally coded this to support booting netbsd images (e.g., AMI amazon) in the cloud which would resize their root according to the disk allocated to them. I haven't had a chance to use it in a while and I believe it's undergone changes/improvements.
> 
> Thanks,
> Chris.

I don't have access to AWS / AMI amazon.

Wouldn't those images still have both disklabel and fdisk?

I don't see any mentions of fdisk or disklabel here, either:

http://cvsweb.netbsd.org/bsdweb.cgi/src/etc/rc.d/resize_root?annotate=1.1&only_with_tag=MAIN

C.

> > On Sep 29, 2019, at 6:58 PM, Constantine A. Murenin <cnst@NetBSD.org> wrote:
> > 
> > On 2019-W38-1 20:08 -0400, Drew DeVault wrote:
> >> Yeah, it seems like resize_root isn't doing the trick. I started
> >> debugging it but if I recall correctly, I had a hard time getting logs
> >> out of it.
> >> 
> > 
> > The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely
> > zero mentions of fdisk or disklabel, so, I don't really understand
> > how it's supposed to work without modifying the partition tables.
> > 
> > I've tried running it in VirtualBox after changing the size of the
> > underlying disk, and, no surprise, the script doesn't do anything at all.
> > 
> > Is there a way to handle fdisk/disklabel in the best way, or to create
> > a bootable i386/amd64 disk image without fdisk/disklabel?
> > 
> > Cc'ing chopps@ and jmcneill@:  do you know how we're supposed to handle
> > fdisk/disklabel change here?
> > 
> > C.
> > 
> 
Christian Hopps
Details
Message ID
<17309D02-E5FB-4A53-95FB-E6A512C04789@chopps.org>
In-Reply-To
<20191001054257.GI8841@Cns.Cns.SU> (view parent)
DKIM signature
missing
Download raw message

> On Oct 1, 2019, at 1:42 AM, Constantine A. Murenin <cnst@NetBSD.org> wrote:
> 
> On 2019-W40-1 20:16 -0400, Christian Hopps wrote:
>> So I originally coded this to support booting netbsd images (e.g., AMI amazon) in the cloud which would resize their root according to the disk allocated to them. I haven't had a chance to use it in a while and I believe it's undergone changes/improvements.
>> 
>> Thanks,
>> Chris.
> 
> I don't have access to AWS / AMI amazon.
> 
> Wouldn't those images still have both disklabel and fdisk?
> 
> I don't see any mentions of fdisk or disklabel here, either:
> 
> http://cvsweb.netbsd.org/bsdweb.cgi/src/etc/rc.d/resize_root?annotate=1.1&only_with_tag=MAIN

https://wiki.netbsd.org/amazon_ec2/build_your_own_ami/

The root file system was a "whole disk" file system.

https://wiki.netbsd.org/amazon_ec2/amis/

Mentions the resize functionality.

Thanks,
Chris.

> 
> C.
> 
>>> On Sep 29, 2019, at 6:58 PM, Constantine A. Murenin <cnst@NetBSD.org> wrote:
>>> 
>>> On 2019-W38-1 20:08 -0400, Drew DeVault wrote:
>>>> Yeah, it seems like resize_root isn't doing the trick. I started
>>>> debugging it but if I recall correctly, I had a hard time getting logs
>>>> out of it.
>>>> 
>>> 
>>> The http://bxr.su/n/etc/rc.d/resize_root script makes absoutely
>>> zero mentions of fdisk or disklabel, so, I don't really understand
>>> how it's supposed to work without modifying the partition tables.
>>> 
>>> I've tried running it in VirtualBox after changing the size of the
>>> underlying disk, and, no surprise, the script doesn't do anything at all.
>>> 
>>> Is there a way to handle fdisk/disklabel in the best way, or to create
>>> a bootable i386/amd64 disk image without fdisk/disklabel?
>>> 
>>> Cc'ing chopps@ and jmcneill@:  do you know how we're supposed to handle
>>> fdisk/disklabel change here?
>>> 
>>> C.
>>> 
>> 
> 
> 
Details
Message ID
<BXJDUTZX0XFP.TV1T2AVFIXTZ@homura>
In-Reply-To
<17309D02-E5FB-4A53-95FB-E6A512C04789@chopps.org> (view parent)
DKIM signature
pass
Download raw message
Hey guys, I've taken your advice and added resize_disklabel=YES and
resize_root=YES to /etc/rc.conf, and copied over the resize_disklabel
script manually. Unfortunately, this seems to have no effect. I have a
NetBSD qcow2 image here which demonstrates the issue:

https://yukari.sr.ht/netbsd.qcow2.xz
Constantine A. Murenin
Details
Message ID
<20191008080234.GN8841@Cns.Cns.SU>
In-Reply-To
<BXJDUTZX0XFP.TV1T2AVFIXTZ@homura> (view parent)
DKIM signature
missing
Download raw message
On 2019-W41-1 11:05 -0400, Drew DeVault wrote:
> Hey guys, I've taken your advice and added resize_disklabel=YES and
> resize_root=YES to /etc/rc.conf, and copied over the resize_disklabel
> script manually. Unfortunately, this seems to have no effect. I have a
> NetBSD qcow2 image here which demonstrates the issue:
> 
> https://yukari.sr.ht/netbsd.qcow2.xz

Have you tried running the resize_disklabel script by itself from shell?  What's the output?

What's the output of fdisk and disklabel?

C.