text-splitters: Add JSFrameworkTextSplitter for Handling JavaScript Framework Code

[BenStuk]

Description

This pull request introduces a new text splitter, JSFrameworkTextSplitter, to the Langchain library. The JSFrameworkTextSplitter extends the RecursiveCharacterTextSplitter to handle JavaScript framework code effectively, including React (JSX), Vue, and Svelte. It identifies and utilizes framework-specific component tags and syntax elements as splitting points, alongside standard JavaScript syntax. This ensures that code is divided at natural boundaries, enhancing the parsing and processing of JavaScript and framework-specific code.

Key Features

• Supports React (JSX), Vue, and Svelte frameworks.
• Identifies and uses framework-specific tags and syntax elements as natural splitting points.
• Extends the existing RecursiveCharacterTextSplitter for seamless integration.

Issue

No specific issue addressed.

Dependencies

No additional dependencies required.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 5, 2025 1:26pm

[ccurme]

Can this be implemented as a new case in the get_separators_for_language method?

[BenStuk]

Can this be implemented as a new case in the get_separators_for_language method?

hi @ccurme, we need the special method to handle React component tags (such as <MyComponent>) in a dynamic way, a fixed list of separators wouldn't be able to recognize them.