How to unzip and zip bzroot

[dlandon 576]

Can someone give me the method of unzipping bzroot and then zipping it back?  I want to make some adjustments and save the changes for testing.

[saarg 429]

I'm not an expert on this, but you might be able to use the part titled "Create new bzroot and copy both bzimage and bzroot to USB flash" from this page on the wiki http://lime-technology.com/wiki/index.php/Building_a_custom_kernel

 

mkdir /mnt/dev/bz-mod
cd /mnt/dev/bz-mod
installpkg /boot/packages/cpio-2.9-i486-2.txz
xzcat ../bzroot | cpio -m -i -d -H newc --no-absolute-filenames
rsync -av --delete /lib/modules/3.9.11p-unRAID/ lib/modules/3.9.11p-unRAID/

#Here's where I add firmware for my hardware
cp /boot/packages/dvb-usb-af9015.fw lib/firmware/

find . | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=preset=9e > /boot/bzroot_new
#Copy bzimage_new
cp arch/x86/boot/bzImage /boot/bzimage_new

 

I had to modify it a little bit when I built my new kernel, but I do not remember what I did and did not document it 

But from what I remember I made a new folder on the cache disk to unpack and pack the bzroot again or else my memory would run out.

 

Hope this helps a little

[dlandon 576]

I think that applies to v5 and not v6.  I believe the packing/zipping mechanism has changed along the way in v6.

[BRiT 308]

I think that applies to v5 and not v6.  I believe the packing/zipping mechanism has changed along the way in v6.

 

cpio works fine for v6. I did that to check out the updates before being able to install them on my server. The only difference if there ever was any would be in what compression level it uses.

[dlandon 576]

I think that applies to v5 and not v6.  I believe the packing/zipping mechanism has changed along the way in v6.

 

cpio works fine for v6. I did that to check out the updates before being able to install them on my server. The only difference if there ever was any would be in what compression level it uses.

 

Ok.  I'll give it a go.

[dlandon 576]

It worked.  I just had to use the 64bit cpio package.

 

installpkg /boot/packages/cpio-2.11-x86_64-2.txz

[saarg 429]

I forgot to say that I did this with version 6, and I figured you would use the 64bit version of cpio 

[BRiT 308]

It worked.  I just had to use the 64bit cpio package.

 

installpkg /boot/packages/cpio-2.11-x86_64-2.txz

 

It's also a part of the NerdPack plugin.

[dlandon 576]

Ah yes.  Thank you.

[jkluch 1]

I wanted to do this too but I'm running into some issues.. and all the documentation for this is out of date at this point.  It looks like bzroot is just a straight cpio archive so no need to use xzcat before the extract.. problem is the only thing in there is kernel/x86/microcode/AuthenticAMD.bin and kernel/x86/microcode/GenuineIntel.bin

 

I'm not sure why those two files are the only things extracting.. I'm not getting any errors from cpio but it's only extracting 3468 blocks.

[saarg 429]

Check this script for how to extract bzroot. 

https://github.com/CHBMB/Unraid-NVIDIA/blob/bassrock-testing/build_scripts/squash-root.sh

[BRiT 308]

The lines dealing with extraction, since bzroot has 2 sections you need to deal with first part and then skip that section to get to the last part:

 

echo -e " ----- Extract BZROOT microcode"

dd if=/boot/bzroot bs=512 count=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') of=bzrootmicrocode

 

echo -e "----- Unpack BZROOT"

dd if=/boot/bzroot bs=512 skip=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

[ghost82 133]

On 1/20/2019 at 2:50 PM, BRiT said:

The lines dealing with extraction, since bzroot has 2 sections you need to deal with first part and then skip that section to get to the last part:

 

 

echo -e " ----- Extract BZROOT microcode"

dd if=/boot/bzroot bs=512 count=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') of=bzrootmicrocode

 

echo -e "----- Unpack BZROOT"

dd if=/boot/bzroot bs=512 skip=$(cpio -ivt -H newc < /boot/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

 

Thank you very much, can you please explain how to put things together?

First command extracts kernel folder with intel/amd microcodes in a cpio archive named microcode, second command extract all the system folders/files.

 

Once changes are applied, how to recreate the cpio bzroot archive?

 

Thank you

 

Update: solved!

Just extract the kernel folder into the folder with the system files, then compress the new bzroot with this command:

 

find . ! -name bzroot | cpio -o -H newc | xz --check=crc32 --x86 --lzma2=preset=9e > /folder-to-bzroot/bzroot

 

lzma2 compression is ok.

Edited June 10, 2019 by ghost82
Solved

[jkluch 1]

On 1/20/2019 at 7:11 AM, saarg said:

Check this script for how to extract bzroot. 

https://github.com/CHBMB/Unraid-NVIDIA/blob/bassrock-testing/build_scripts/squash-root.sh

Hey @CHBMB Does the code that use to live here still exist somewhere as reference?  I see that this code is gone and now there are linuxserver unraid plugins that exists with prebuilt archives.  This is nice/convenient for a lot of unraid users but removing the source used to create the archives and making users trust the prebuilt archives is kind of a shame.  It was useful to refer to those scripts when I was making changes of my own (I wouldn't have been able to figure out how to unpackage & modify these bz files on my own).  Is there a reason the source scripts for these aren't included along with the plugin or perhaps elsewhere? (I'm specifically referring to these two projects Unraid-DVB-Plugin and Unraid-Nvidia-Plugin)

[BRiT 308]

Have you looked under here? https://github.com/linuxserver/Unraid-DVB/tree/master/build_scripts

 

In particular https://github.com/linuxserver/Unraid-DVB/blob/master/build_scripts/kernel-compile-module.sh

 

 

Edited June 11, 2019 by BRiT

[CHBMB 976]

25 minutes ago, jkluch said:

Hey @CHBMB Does the code that use to live here still exist somewhere as reference?  I see that this code is gone and now there are linuxserver unraid plugins that exists with prebuilt archives.  This is nice/convenient for a lot of unraid users but removing the source used to create the archives and making users trust the prebuilt archives is kind of a shame.  It was useful to refer to those scripts when I was making changes of my own (I wouldn't have been able to figure out how to unpackage & modify these bz files on my own).  Is there a reason the source scripts for these aren't included along with the plugin or perhaps elsewhere? (I'm specifically referring to these two projects Unraid-DVB-Plugin and Unraid-Nvidia-Plugin)

The DVB source scripts are here.  https://github.com/linuxserver/Unraid-DVB

 

The Nvidia source scripts we deliberately are keeping closed as we're a tiny bit scared of Nvidia and the possibility of people using them to circumvent certain Nvidia restrictions.

[CHBMB 976]

Here's how to uncompress/compress bzroot

 

 ##Extract microcode

dd if=$D/unraid/bzroot bs=512 count=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') of=$D/unraid/bzmicrocode

##Unpack

dd if=$D/unraid/bzroot bs=512 skip=$(cpio -ivt -H newc < $D/unraid/bzroot 2>&1 > /dev/null | awk '{print $1}') | xzcat | cpio -i -d -H newc --no-absolute-filenames

##Prepend microcode and Compress

cp $D/unraid/bzmicrocode $D/$VERSION/nvidia/bzroot

find . | cpio -o -H newc | xz --format=lzma >> $D/$VERSION/nvidia/bzroot 

 

[aterfax 9]

On 6/11/2019 at 1:28 AM, CHBMB said:

The DVB source scripts are here.  https://github.com/linuxserver/Unraid-DVB

 

The Nvidia source scripts we deliberately are keeping closed as we're a tiny bit scared of Nvidia and the possibility of people using them to circumvent certain Nvidia restrictions.

I'm trying to compile with support for both nvidia and turning on CONFIG_NETFILTER_XT_MATCH_IPVS in order to support docker swarm.

I know you probably want to keep them closed source to prevent circumvention of nvidia restrictions but I can tell you through testing. Having access to your kernel compile scripts is not needed to achieve this.

Could you open source the scripts or give me a hand in the order of operations in order to compile getting this working?

At the moment this is proving problematic due to the lack of the nvidia modules in the compiled output.

Edit: not the only one apparently -

 

Edited July 24, 2019 by aterfax