Tweak symbol comparison logic a bit #71

Job	Run time
Check clippy for Python bindings	34s
sdist	20s
linux (x86_64)	1m 25s
macos (x86_64)	51s
linux (x86)	1m 46s
linux (aarch64)	1m 38s
macos (aarch64)	48s
linux (armv7)	1m 46s
windows (x64)	1m 39s
windows (x86)	1m 46s
linux (s390x)	1m 48s
linux (ppc64le)	1m 37s
Release	0s
	15m 58s

Provide feedback