How to unzip and zip bzroot


dlandon

Recommended Posts

I'm not an expert on this, but you might be able to use the part titled "Create new bzroot and copy both bzimage and bzroot to USB flash" from this page on the wiki http://lime-technology.com/wiki/index.php/Building_a_custom_kernel

 

mkdir /mnt/dev/bz-mod
cd /mnt/dev/bz-mod
installpkg /boot/packages/cpio-2.9-i486-2.txz
xzcat ../bzroot | cpio -m -i -d -H newc --no-absolute-filenames
rsync -av --delete /lib/modules/3.9.11p-unRAID/ lib/modules/3.9.11p-unRAID/

#Here's where I add firmware for my hardware
cp /boot/packages/dvb-usb-af9015.fw lib/firmware/

find . | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=preset=9e > /boot/bzroot_new
#Copy bzimage_new
cp arch/x86/boot/bzImage /boot/bzimage_new

 

I had to modify it a little bit when I built my new kernel, but I do not remember what I did and did not document it  ::)

But from what I remember I made a new folder on the cache disk to unpack and pack the bzroot again or else my memory would run out.

 

Hope this helps a little :)

  • Like 1
Link to comment

I think that applies to v5 and not v6.  I believe the packing/zipping mechanism has changed along the way in v6.

 

cpio works fine for v6. I did that to check out the updates before being able to install them on my server. The only difference if there ever was any would be in what compression level it uses.

Link to comment

I think that applies to v5 and not v6.  I believe the packing/zipping mechanism has changed along the way in v6.

 

cpio works fine for v6. I did that to check out the updates before being able to install them on my server. The only difference if there ever was any would be in what compression level it uses.

 

Ok.  I'll give it a go.

Link to comment
  • 3 years later...

I wanted to do this too but I'm running into some issues.. and all the documentation for this is out of date at this point.  It looks like bzroot is just a straight cpio archive so no need to use xzcat before the extract.. problem is the only thing in there is kernel/x86/microcode/AuthenticAMD.bin and kernel/x86/microcode/GenuineIntel.bin

 

I'm not sure why those two files are the only things extracting.. I'm not getting any errors from cpio but it's only extracting 3468 blocks.

Link to comment

The lines dealing with extraction, since bzroot has 2 sections you need to deal with first part and then skip that section to get to the last part:

 


echo -e " ----- Extract BZROOT microcode"

dd if=/boot/bzroot bs=512 count=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') of=bzrootmicrocode

 

echo -e "----- Unpack BZROOT"

dd if=/boot/bzroot bs=512 skip=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

  • Like 2
Link to comment
  • 4 months later...
On 1/20/2019 at 2:50 PM, BRiT said:

The lines dealing with extraction, since bzroot has 2 sections you need to deal with first part and then skip that section to get to the last part:

 

 


echo -e " ----- Extract BZROOT microcode"

dd if=/boot/bzroot bs=512 count=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') of=bzrootmicrocode

 

echo -e "----- Unpack BZROOT"

dd if=/boot/bzroot bs=512 skip=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

 

Thank you very much, can you please explain how to put things together?

First command extracts kernel folder with intel/amd microcodes in a cpio archive named microcode, second command extract all the system folders/files.

 

Once changes are applied, how to recreate the cpio bzroot archive?

 

Thank you

 

Update: solved!

Just extract the kernel folder into the folder with the system files, then compress the new bzroot with this command:

 

find . ! -name bzroot | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=preset=9e > /folder-to-bzroot/bzroot

 

lzma2 compression is ok.

Edited by ghost82
Solved
Link to comment
On 1/20/2019 at 7:11 AM, saarg said:

Hey @CHBMB Does the code that use to live here still exist somewhere as reference?  I see that this code is gone and now there are linuxserver unraid plugins that exists with prebuilt archives.  This is nice/convenient for a lot of unraid users but removing the source used to create the archives and making users trust the prebuilt archives is kind of a shame.  It was useful to refer to those scripts when I was making changes of my own (I wouldn't have been able to figure out how to unpackage & modify these bz files on my own).  Is there a reason the source scripts for these aren't included along with the plugin or perhaps elsewhere? (I'm specifically referring to these two projects Unraid-DVB-Plugin and Unraid-Nvidia-Plugin)

Link to comment
25 minutes ago, jkluch said:

Hey @CHBMB Does the code that use to live here still exist somewhere as reference?  I see that this code is gone and now there are linuxserver unraid plugins that exists with prebuilt archives.  This is nice/convenient for a lot of unraid users but removing the source used to create the archives and making users trust the prebuilt archives is kind of a shame.  It was useful to refer to those scripts when I was making changes of my own (I wouldn't have been able to figure out how to unpackage & modify these bz files on my own).  Is there a reason the source scripts for these aren't included along with the plugin or perhaps elsewhere? (I'm specifically referring to these two projects Unraid-DVB-Plugin and Unraid-Nvidia-Plugin)

The DVB source scripts are here.  https://github.com/linuxserver/Unraid-DVB

 

The Nvidia source scripts we deliberately are keeping closed as we're a tiny bit scared of Nvidia and the possibility of people using them to circumvent certain Nvidia restrictions.

Link to comment

Here's how to uncompress/compress bzroot

 

 ##Extract microcode

dd if=$D/unraid/bzroot bs=512 count=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') of=$D/unraid/bzmicrocode

##Unpack

dd if=$D/unraid/bzroot bs=512 skip=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

##Prepend microcode and Compress

cp $D/unraid/bzmicrocode $D/$VERSION/nvidia/bzroot

find . | cpio -o -H newc | xz --format=lzma >> $D/$VERSION/nvidia/bzroot 

 

  • Like 1
  • Upvote 1
Link to comment
  • 1 month later...
On 6/11/2019 at 1:28 AM, CHBMB said:

The DVB source scripts are here.  https://github.com/linuxserver/Unraid-DVB

 

The Nvidia source scripts we deliberately are keeping closed as we're a tiny bit scared of Nvidia and the possibility of people using them to circumvent certain Nvidia restrictions.

I'm trying to compile with support for both nvidia and turning on CONFIG_NETFILTER_XT_MATCH_IPVS in order to support docker swarm.

I know you probably want to keep them closed source to prevent circumvention of nvidia restrictions but I can tell you through testing. Having access to your kernel compile scripts is not needed to achieve this.

Could you open source the scripts or give me a hand in the order of operations in order to compile getting this working?

At the moment this is proving problematic due to the lack of the nvidia modules in the compiled output.

Edit: not the only one apparently -

 

Edited by aterfax
Link to comment
  • 1 year later...

Hello there,

 

I will renew the activity in this thread a little bit.

Is there someone able to replicate the "UNZIPPING" root file system and the "ZIPPING" it back remaining still bootable even without changes?

I am struggling with it last few days, and looks like unRaid has changed the method of building the root file for initramfs.

It is not only the early Kernel and root folders(compressed), but looks like there is Kernel + root folders + other uknown stuff.

 

Your help would be nice, or at least some ideas.

 

bzroot file inspection output.:

homelab@homelab-DH310V2:~/Desktop/stock/bzroot_folder$ binwalk bzroot_orig

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             ASCII cpio archive (SVR4 with no CRC), file name: "."
112           0x70            ASCII cpio archive (SVR4 with no CRC), file name: "kernel"
232           0xE8            ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86"
356           0x164           ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86/microcode",
488           0x1E8           ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86/microcode/GenuineIntel.bin"
3624572       0x374E7C        ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86/microcode/AuthenticAMD.bin"
3655268       0x37C664        ASCII cpio archive (SVR4 with no CRC), file name: "TRAILER!!!"
3655680       0x37C800        xz compressed data
14747435      0xE1072B        EBML file
27330002      0x1A105D2       mcrypt 2.5 encrypted data, algorithm: "-", keysize: 14961 bytes, mode: "V",
43539972      0x2985E04       Unix path: /usr/local/go/src/runtime/runtime-gdb.py
58338583      0x37A2D17       Zlib compressed data, default compression
79135327      0x4B7825F       PARity archive data - file number 5362
93608855      0x5945B97       ASCII cpio archive (SVR4 with no CRC)
125494792     0x77AE608       MPEG transport stream data
139663506     0x8531892       Zlib compressed data, default compression
139663558     0x85318C6       Zlib compressed data, default compression
139663590     0x85318E6       Zlib compressed data, default compression
139664346     0x8531BDA       Zlib compressed data, default compression
139666162     0x85322F2       Zlib compressed data, default compression
139666194     0x8532312       Zlib compressed data, default compression
139666774     0x8532556       Zlib compressed data, default compression
139670750     0x85334DE       ASCII cpio archive (SVR4 with no CRC)
140843025     0x8651811       gzip compressed data, last modified: 2082-09-03 16:48:36 (bogus date)

 

Link to comment
On 6/11/2019 at 3:34 AM, CHBMB said:

Here's how to uncompress/compress bzroot

 


 ##Extract microcode

dd if=$D/unraid/bzroot bs=512 count=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') of=$D/unraid/bzmicrocode

##Unpack

dd if=$D/unraid/bzroot bs=512 skip=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

##Prepend microcode and Compress

cp $D/unraid/bzmicrocode $D/$VERSION/nvidia/bzroot

find . | cpio -o -H newc | xz --format=lzma >> $D/$VERSION/nvidia/bzroot 

 

 

 

is it still working pack and unpack bzroot? i wanted to try but have some errors in command line ...

can i use it in command line or user script ?

Link to comment
  • 3 weeks later...
On 4/11/2021 at 10:18 PM, pafiiel said:

Hello there,

 

I will renew the activity in this thread a little bit.

Is there someone able to replicate the "UNZIPPING" root file system and the "ZIPPING" it back remaining still bootable even without changes?

I am struggling with it last few days, and looks like unRaid has changed the method of building the root file for initramfs.

It is not only the early Kernel and root folders(compressed), but looks like there is Kernel + root folders + other uknown stuff.

 

Your help would be nice, or at least some ideas.

 

bzroot file inspection output.:


homelab@homelab-DH310V2:~/Desktop/stock/bzroot_folder$ binwalk bzroot_orig

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             ASCII cpio archive (SVR4 with no CRC), file name: "."
112           0x70            ASCII cpio archive (SVR4 with no CRC), file name: "kernel"
232           0xE8            ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86"
356           0x164           ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86/microcode",
488           0x1E8           ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86/microcode/GenuineIntel.bin"
3624572       0x374E7C        ASCII cpio archive (SVR4 with no CRC), file name: "kernel/x86/microcode/AuthenticAMD.bin"
3655268       0x37C664        ASCII cpio archive (SVR4 with no CRC), file name: "TRAILER!!!"
3655680       0x37C800        xz compressed data
14747435      0xE1072B        EBML file
27330002      0x1A105D2       mcrypt 2.5 encrypted data, algorithm: "-", keysize: 14961 bytes, mode: "V",
43539972      0x2985E04       Unix path: /usr/local/go/src/runtime/runtime-gdb.py
58338583      0x37A2D17       Zlib compressed data, default compression
79135327      0x4B7825F       PARity archive data - file number 5362
93608855      0x5945B97       ASCII cpio archive (SVR4 with no CRC)
125494792     0x77AE608       MPEG transport stream data
139663506     0x8531892       Zlib compressed data, default compression
139663558     0x85318C6       Zlib compressed data, default compression
139663590     0x85318E6       Zlib compressed data, default compression
139664346     0x8531BDA       Zlib compressed data, default compression
139666162     0x85322F2       Zlib compressed data, default compression
139666194     0x8532312       Zlib compressed data, default compression
139666774     0x8532556       Zlib compressed data, default compression
139670750     0x85334DE       ASCII cpio archive (SVR4 with no CRC)
140843025     0x8651811       gzip compressed data, last modified: 2082-09-03 16:48:36 (bogus date)

 

 

On 6/11/2019 at 3:34 AM, CHBMB said:

Here's how to uncompress/compress bzroot

 


 ##Extract microcode

dd if=$D/unraid/bzroot bs=512 count=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') of=$D/unraid/bzmicrocode

##Unpack

dd if=$D/unraid/bzroot bs=512 skip=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

##Prepend microcode and Compress

cp $D/unraid/bzmicrocode $D/$VERSION/nvidia/bzroot

find . | cpio -o -H newc | xz --format=lzma >> $D/$VERSION/nvidia/bzroot 

 

 

On 2/28/2015 at 2:43 PM, saarg said:

I'm not an expert on this, but you might be able to use the part titled "Create new bzroot and copy both bzimage and bzroot to USB flash" from this page on the wiki http://lime-technology.com/wiki/index.php/Building_a_custom_kernel

 

 


mkdir /mnt/dev/bz-mod
cd /mnt/dev/bz-mod
installpkg /boot/packages/cpio-2.9-i486-2.txz
xzcat ../bzroot | cpio -m -i -d -H newc --no-absolute-filenames
rsync -av --delete /lib/modules/3.9.11p-unRAID/ lib/modules/3.9.11p-unRAID/

#Here's where I add firmware for my hardware
cp /boot/packages/dvb-usb-af9015.fw lib/firmware/

find . | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=preset=9e > /boot/bzroot_new
#Copy bzimage_new
cp arch/x86/boot/bzImage /boot/bzimage_new
 

 

 

I had to modify it a little bit when I built my new kernel, but I do not remember what I did and did not document it  ::)

But from what I remember I made a new folder on the cache disk to unpack and pack the bzroot again or else my memory would run out.

 

Hope this helps a little :)

 

 

Finally i unpacked and packed bzroot without changes and bzroot check OK but in the middle of boot i have a problem :

 INIT: version 2.97 booting 

grep: /proc/filesystems: no such file or directory

        /proc/mounts: no such file or directory

      ans some kinds of other mounts

Maybe Some one know where the problem ?

Link to comment
  • 2 years later...
On 5/3/2021 at 11:13 AM, Masterwishx said:

Finally i unpacked and packed bzroot without changes and bzroot check OK but in the middle of boot i have a problem :

 INIT: version 2.97 booting 

grep: /proc/filesystems: no such file or directory

        /proc/mounts: no such file or directory

      ans some kinds of other mounts

Maybe Some one know where the problem ?

In case others who are trying to make changes to bzroot stumble on this thread, I had the same problem as @Masterwishx did (see above) and was able to solve it by removing the "--no-absolute-filenames" cpio option when unpacking the initramfs. It appears that as of at least Unraid 6.10+ the initramfs cpio archive uses relative paths and relies on some relative symlinks (e.g. ../../<link>) which are incorrectly altered if the archive is expanded with "--no-absolute-filenames". A working procedure for modifying bzroot on Unraid 6.10+ might look something like this:

 

##Extract microcode

dd if=/boot/bzroot bs=512 count=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') of=/tmp/bzmicrocode

##Unpack
mkdir /tmp/unpacked-bzroot
cd /tmp/unpacked-bzroot
dd if=/boot/bzroot bs=512 skip=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc

##Make desired modifications to the expanded bzroot filesystem in /tmp/unpacked-bzroot

##Prepend microcode and Compress
cd /tmp/unpacked-bzroot
find . | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=preset=9e > /tmp/bzroot_new
cat /tmp/bzmicrocode /tmp/bzroot_new > /tmp/bzroot
sha256sum /tmp/bzroot > /tmp/bzroot.sha256 

##Finally edit bzroot.sha256 to remove the trailing filename and copy the new files into /boot

 

Edited by pants
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.