ICU-22984 Generate old monkeys

[eggrobin]

C++ only for now, I will do the old monkeys from Java separately (that will be more of the same but with subtle differences, as we will need to expand the UnicodeSets ourselves before feeding them to Pattern). This uses ICU regexes to match the context before and after, or to match the left-hand side of a remap rule, as in UAX14 and UAX29. The regexes are the ones used to generate the conformance tests in the Unicode tools; they now closely match the ones in the UAX.
Also line breaking only; the other ones will be very similar (and simpler).

The generated partition and rules are from unicode-org/unicodetools#979. Note that the generation involves transforming & and - (UnicodeSet syntax) to && and -- (ICU regex character class syntax).

Checklist

• Required: Issue filed: ICU-22984
• Required: The PR title must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
• Required: The PR description must include the link to the Jira Issue, for example by completing the URL in the first checklist item
• Required: Each commit message must be prefixed with a JIRA Issue number. Example: "ICU-1234 Fix xyz"
• Issue accepted (done by Technical Committee after discussion)
• Tests included, if applicable
• API docs and/or User Guide docs changed or added, if applicable

ALLOW_MANY_COMMITS=true

[macchiati]

Looks like a good optimization (and cleanup)

[markusicu]

very nice replacement of hardcoded rules with generated ones!

[markusicu]

I think that's because . matches CR LF, so it won't match just half of that.

[eggrobin]

That may well be the explanation, but even with regex greed, that still feels like a bug: /(rn|[a-z])*n/ still matches rn.

[markusicu]

great comments!

[markusicu]

should the final offset be -2?

[eggrobin]

No, the offset variable is with respect to the previous value of indexInRemapped, so 6 + -1 + -1 = 4.

[markusicu]

Remember to squash the commits. It's ok if you want to merge multiple commits if they make sense as separate ones.
Each one's message needs to start with ICU-22984
Squashing without contents change will not require re-approval.

[jira-pull-request-webhook]

Hooray! The files in the branch are the same across the force-push. 😃

~ Your Friendly Jira-GitHub PR Checker Bot

[eggrobin]

It's ok if you want to merge multiple commits if they make sense as separate ones.

I tried with four steps (unicodetools-like implementation, optimization, code motion which wants to be on its own because it is undiffable, post-motion cleanup). Let me know if you like that, otherwise I can squish it all into one commit.

[markusicu]

It's ok if you want to merge multiple commits if they make sense as separate ones.

I tried with four steps (unicodetools-like implementation, optimization, code motion which wants to be on its own because it is undiffable, post-motion cleanup). Let me know if you like that, otherwise I can squish it all into one commit.

wfm tnx
I added the allow-many-commits magic into the description.

[FrankYFTang]

The Exhaustive Tests for ICU #22 is broken between e025466 and 2e57f07 in TestMonkey

https://github.com/unicode-org/icu/actions/runs/12209763924

Likely caused by this PR

[eggrobin]

Yes, fixing in #3296.