Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datax 增量同步数据问题 #686

Open
demo3025 opened this issue Jan 14, 2025 · 0 comments
Open

datax 增量同步数据问题 #686

demo3025 opened this issue Jan 14, 2025 · 0 comments
Labels
question Further information is requested

Comments

@demo3025
Copy link

你好,我想请问下,datax 如何增量以 orc 格式的数据同步到 hdfs 中,当我使用 datax 同步数据以 orc 格式存储时(append 模式),每次都会生成一个指定目录指定 filename 拼接随机 id 的一个 orc 文件,看起来并不能追加;
我的场景是这样,定时同步订单数据,当订单数据状态发生变化的时候同步反应到 hdfs 的 orc 文件中去,我的预想是,全量同步数据,然后定时增量同步,当订单发生变化时,能否在 orc 文件中删除或者更新这笔订单记录(如状态变更)

@demo3025 demo3025 added the question Further information is requested label Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant