Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] Add encoding module in lib_ccxr #1628

Merged

Conversation

IshanGrover2004
Copy link
Contributor

In raising this pull request, I confirm the following (please check boxes):

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.
  • I have mentioned this change in the changelog.

My familiarity with the project is as follows (check one):

  • I have never used CCExtractor.
  • I have used CCExtractor just a couple of times.
  • I absolutely love CCExtractor, but have not contributed previously.
  • I am an active contributor to CCExtractor.

Closes #1554

Added a module in lib_ccxr for working with different kinds of text encoding formats.
This PR adds types used for representing Strings encoded in a certain format like Line 21 or UCS-2 and the ability to interconvert between them.

A module for working with different kinds of text encoding formats
Comment on lines 569 to 571
fn latin1_to_line21(_c: Latin1Char) -> Line21Char {
todo!()
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 todo remaining
Researching on this one
@PunitLodha @elbertronnie Any idea about it, This function is not included in C but rust match(line 417) needed it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though it is not required in C, I wanted all forms to be convertable to all other forms of String simply because of completeness. Implementing this would take some time which is why I left a todo in there to do it later.

If you want to implement it you can do it by inverting the logic in line21_to_latin1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though it is not required in C, I wanted all forms to be convertable to all other forms of String simply because of completeness. Implementing this would take some time which is why I left a todo in there to do it later.

Yeah exactly @elbertronnie , I removed some todo from your code, just left with this one only

If you want to implement it you can do it by inverting the logic in line21_to_latin1.

I am trying to do that only

}

fn line21_to_utf8(c: Line21Char) -> char {
0x80 as char
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation feels incorrect. It will always return 0x80. Is this how it is defined in C?

Copy link
Contributor Author

@IshanGrover2004 IshanGrover2004 Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh Sorry that is mistake

}

fn ucs2_to_line21(c: Ucs2Char) -> Line21Char {
if c < 0x80 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation is also wrong. It should be having inverted logic of line21_to_ucs2. But to create this, you will require latin1_to_line21 first. I would recommend to either implement this or put a todo here. It is not used in C anyway.

I know that I wrote this and I don't remember what I was thinking while writing this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done it @elbertronnie by reversing the function

@IshanGrover2004
Copy link
Contributor Author

@PunitLodha @elbertronnie Can you now see it & review

@PunitLodha PunitLodha merged commit 5f9b395 into CCExtractor:master Aug 10, 2024
13 of 15 checks passed
@PunitLodha PunitLodha mentioned this pull request Aug 10, 2024
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants