-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Symlink Troubles #761
Comments
After taking another look at this, I finally tracked down the cause of issue 3 and updated the text above. I now think the series of patches above would work reasonably well, but I'm sure they could be better engineered. I can open a PR with these changes if there's interest, just let me know. |
@AndrewFasano yes open a PR, ideally with one commit per issue you're fixing. We'll review it asap cause these are major issues. |
Changes to romfs and yaffs would also be required. The whole thing worked because arguments were swapped on both ends ... diff --git a/unblob/handlers/filesystem/romfs.py b/unblob/handlers/filesystem/romfs.py
index bfce398..0fd3e99 100644
--- a/unblob/handlers/filesystem/romfs.py
+++ b/unblob/handlers/filesystem/romfs.py
@@ -255,7 +255,7 @@ class RomFSHeader:
def create_symlink(self, output_path: Path, inode: FileHeader):
target_path = Path(inode.content.decode("utf-8"))
- self.fs.create_symlink(src=target_path, dst=output_path)
+ self.fs.create_symlink(src=output_path, dst=target_path)
def create_hardlink(self, output_path: Path, inode: FileHeader):
if inode.spec_info in self.inodes:
diff --git a/unblob/handlers/filesystem/yaffs.py b/unblob/handlers/filesystem/yaffs.py
index 966c8ea..a2b0267 100644
--- a/unblob/handlers/filesystem/yaffs.py
+++ b/unblob/handlers/filesystem/yaffs.py
@@ -500,7 +500,7 @@ class YAFFSParser:
elif entry.object_type == YaffsObjectType.FILE:
fs.write_chunks(out_path, self.get_file_chunks(entry))
elif entry.object_type == YaffsObjectType.SYMLINK:
- fs.create_symlink(src=Path(entry.alias), dst=out_path)
+ fs.create_symlink(src=out_path, dst=Path(entry.alias))
elif entry.object_type == YaffsObjectType.HARDLINK:
dst_entry = self.file_entries[entry.equiv_id].data
dst_path = self.resolve_path(dst_entry) |
Paging @e3krisztian since you worked on Filesystem API. |
I'm still pretty confused by all this so I'm trying to building some test to mimic the failures I saw with the CPIO archive in that firmware image. I'll report back when I have more information / am more confident in these fixes. |
I have commented on the MR, but is more relevant here: FileSystem.create_symlink(src, dst) is supposed to be following the unix command line order and naming
Except these are named differently in the
So I think the way to go would be to fix the affected handlers (cpio), and rename the arguments to be less confusing. |
@e3krisztian we can split the fixes in smaller PR:
|
After looking into the PR for this issue, I have decided to verify the original issue and extracted the linked firmware with the current There is 2e6a7ae merged, maybe that was the problem here, and they are fixed by that as well. I see no trivially visible problem with the symlinks, and even the mentioned $ ll DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/arp
lrwxrwxrwx 1 ?? ?? 14 febr 14 18:20 DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/arp -> ../bin/busybox while there is no "Potential path traversal through link" reported. Full list of `busybox` links$ find DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/ -ls | rg busybox | cut -c 74- | sort DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/ash -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/cat -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/chmod -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/cp -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/date -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/echo -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/grep -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/kill -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/login -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/ls -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/mkdir -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/mknod -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/mount -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/ping6 -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/ping -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/ps -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/pwd -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/rm -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/sed -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/sh -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/sleep -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/touch -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/umount -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/bin/vi -> busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/init -> bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/arp -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/halt -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/ifconfig -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/init -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/insmod -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/lsmod -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/mdev -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/poweroff -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/reboot -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/rmmod -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/route -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/syslogd -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/udhcpc -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/sbin/zcip -> ../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/arping -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/[ -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/[[ -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/expr -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/free -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/ftpd -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/ftpputimage -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/ftpputvideo -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/killall -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/printf -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/test -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/top -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/bin/uptime -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/sbin/brctl -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/sbin/chpasswd -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/sbin/inetd -> ../../bin/busybox DCS-5009L_REVA_FIRMWARE_1.00.B1.zip_extract/dcs5009l_v100_b1.bin_extract/327744-7144690.lzma_extract/lzma.uncompressed_extract/3813376-9394347.lzma_extract/lzma.uncompressed_extract/usr/sbin/telnetd -> ../../bin/busybox @AndrewFasano could you please re-validate your findings with the current unblob |
Oh no, I'm also not able to reproduce this issue! I may have shot myself in the foot here - the (incorrect) changes to swap src/dst seem to have broken a bunch of other things. Prior to 2e6a7ae this filesystem couldn't be extracted at all, so I was making some source modifications to try getting it to work - in the process I probably introduced these issues. Sorry about that, I'm going to close this issue. There are definitely still some symlink issues (that trigger with the current head of main), but this doesn't seem to be one of them. |
I think I figured out the root cause of my confusion! I was running with this draft PR which introduced a backwards check where it would refuse to create a symlink if the destination existed. Then I went down a long and incorrect path of swapping arguments everywhere else, but not there. |
I believe there are a handful of bugs in how unblob handles symlinks. I'm not too confident about my understanding of unblob internals so I wanted to talk through these instead of just proposing fixes.
I've encountered all these failures with this firmware image: https://legacyfiles.us.dlink.com/DCS-5009L/REVA/FIRMWARE/DCS-5009L_REVA_FIRMWARE_1.00.B1.zip. I'm comparing the unblob output to that generated by running binwalk.
Issue 1: CPIO extractor calls
create_symlink
with backwards arguments - the source of a symlink is the filename (entry.path) while the destination is where the link points to (I think). Without this fix all the binaries in this firmware that symlink to/bin/busybox
are missing in the output archive (i.e.,/sbin/arp
) as_get_checked_link
in file_utils.py sees that/bin/busybox
already exists and skips (checking the destination, not the source, since the arguments are swapped).Issue 2:
create_symlink
creates symlinks backwards, they currently point fromdst
tosrc
. The call to_get_checked_link
has it right and correctly checks if thesrc
argument already exists. But, the code to actually create the link would previously create a link fromdst -> src
instead of fromsrc -> dst
. This would raise aFileExistsError
if Issue 1 is fixed with no other changes.Issue 3: False positives in path traversal detection. This firmware contains a CPIO archive with some valid, non-malicious symlinks, but they're incorrectly flagged as potential path traversals and skipped during extraction. This one has me pretty confused and I'm not sure what the best fix would be.
If I run
cpio -itv
on the CPIO archive, I see a valid symlink for init to busybox:But during unblob extraction, I get a warning and the symlink is skipped:
This happens for ~20 binaries in this firmware that have relative symlink to
busybox
that involve..
as part of the paths. I believe this is in part caused by the FSLink constructor failing to createself.dst
based on theself.src
path, which can maybe be fixed with:With this fix, all the files except 2 (e.g.,
etc_ro/ppp/peers/3g -> /etc/3g
)are then created successfully - these remaining 2 have end up with..
in the src field incorrectly after the body of theis_absolute
check increate_symlink
which incorrectly checkssrc
instead ofdst
. This can be fixed with:The text was updated successfully, but these errors were encountered: