[x-] is not correctly handled

[danny0838]

Prerequisites

• I verified that this is not a filter list issue. Report any issues with filter lists or broken website functionality in the uAssets issue tracker.
• This is not a support issue or a question. For any support, questions or help, visit /r/uBlockOrigin.
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue.
• The issue is not present after disabling uBO in the browser.
• I checked the documentation to understand that the issue I am reporting is not normal behavior.

I tried to reproduce the issue when...

• uBO is the only extension.
• uBO uses default lists and settings.
• using a new, unmodified browser profile.

Description

The revised regex analyzer does not correctly handle [x-], which is treated as [\-] and thus x is not calculated during token extraction.

I have raised an issue to the library developer: foo123/RegexAnalyzer#5. Before the library is fixed, we can preprocess the regex string to escape problem-making "-"s in a character group before sending it to the analyzer:

const REGEX_RE_STR_FIXER = /\\[\s\S]|\[([^\\\]]*(?:\\[\S\s][^\\\]]*)*)\]/g;
const REGEX_RE_STR_FIXER_FUNC = (m, cg) => (cg ? '[' + cg.replace(REGEX_RE_STR_FIXER_CG, REGEX_RE_STR_FIXER_CG_FUNC) + ']' : m);
const REGEX_RE_STR_FIXER_CG = /^\^|(?:[^\\\]]|\\(?:x[0-9A-Fa-f]{2}|u[0-9A-Fa-f]{4}|[^dDsSwW]))-(?:[^\\\]]|\\(?:x[0-9A-Fa-f]{2}|u[0-9A-Fa-f]{4}|[^dDsSwW]))|\\[\s\S]|(-)/g;
const REGEX_RE_STR_FIXER_CG_FUNC = (m, h) => (h ? '\\-' : m);

// fix analyzer bug for e.g. [x-], [-x], [\d-x], [x-\d]
// https://github.com/foo123/RegexAnalyzer/issues/5
reStr = reStr.replace(REGEX_RE_STR_FIXER, REGEX_RE_STR_FIXER_FUNC);

This is somehow ugly and imperformant, but it may be the only way if we're not going to implement a full fix of the library.

Also note that this does not fix the bug in 5th case yet. Fortunately the case should not exist in a real-world URL filter.

A specific URL where the issue occurs.

https://example.com/xfoo

Steps to Reproduce

1. Add a user rule /^https?://(?:example\.com|ex\.com)/[x-]foo\b/$doc
2. Visit the URL

Expected behavior

The page should be blocked.

Actual behavior

The page is not blocked.

uBO version

1.45.3b4

Browser name and version

Independant

Operating System and version

Independant

[gwarser]

Maybe this is why regex was reported as invalid when - was last in the group?

[danny0838]

@gwarser Where is it reported?

ECMAscript spec seems to allow such use case.

[gwarser]

AdguardTeam/AdguardFilters#134630

[danny0838]

@gwarser I have read the thread but I don't think there is a consensus that trailing - is invalid. I think it is the app/library that has to support it as browsers and the spec support it.

[gwarser]

I mean, it's this library limitation.

[gorhill]

Any more of these cases with regex tokenizer, please add here instead of opening new issue.

[danny0838]

The developer of Regex.js is fixing the issues and it seems that this issue is mostly resolved by the latest release (with a bug that may happen in a very rare case). Maybe we can simply wait for a while for his working.

[danny0838]

Issues in foo123/RegexAnalyzer#5 seems resolved in 1.2.0

[gorhill]

Version of 1.2.0 was imported.

[micsthepick]

Syntax highlighting is confusing:

/ex-ample\.com/ has no coloration
/example\.com/ has coloration
and yet no message to explain why.

[gorhill]

/ex-ample\.com/ has no "error"
/example\.com/ has 1 "error"

Orange is not an error, it just means no token can be extracted from the regex, and as such the filter will be less efficient.

A token can be extracted from the first one, ample. No token can be extracted from the second one.

[gorhill]

The ultimate revision is the one based on a regex analyzer, introduced in gorhill/uBlock@426395a.