Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

INTERNAL: read while PIPE_ERROR received in the pipe operation #839

Merged
merged 1 commit into from
Jan 9, 2025

Conversation

oliviarla
Copy link
Collaborator

@oliviarla oliviarla commented Nov 15, 2024

πŸ”— Related Issue

⌨️ What I did

  • λ³Έ PRμ—μ„œλŠ” νŒŒμ΄ν”„ μ—°μ‚° μš”μ²­μ—μ„œ μ—λŸ¬(CLIENT_ERROR, ERROR, SERVER_ERROR) 응닡 μ‹œ PIPE_ERRORλ₯Ό λκΉŒμ§€ 읽도둝 λ³€κ²½ν•œλ‹€.
  • κΈ°μ‘΄μ—λŠ” μ—λŸ¬ 응닡 μ‹œ μ¦‰μ‹œ μ˜ˆμ™Έλ₯Ό 생성해 던쑌기 λ•Œλ¬Έμ— PIPE_ERRORλ₯Ό 읽지 μ•Šμ•˜λ‹€.
  • 동기 νŒŒμ΄ν”„ μ—°μ‚° 처리 μ‹œ μ—λŸ¬λ‘œ 인해 μˆ˜ν–‰λ˜μ§€ μ•Šμ€ 연산을 failedResultλ₯Ό 톡해 μ•Œλ €μ•Ό ν•˜λŠ”λ°, κΈ°μ‘΄ ν˜•νƒœμ—μ„œλŠ” μ—λŸ¬ μ‹œ λ°”λ‘œ μ˜ˆμ™Έλ₯Ό λ˜μ§€λ―€λ‘œ, μ—λŸ¬ λ°œμƒμœΌλ‘œ 인해 μˆ˜ν–‰λ˜μ§€ μ•Šμ€ 연산에 λŒ€ν•΄ callback.gotStatus()λ₯Ό ν˜ΈμΆœν•  수 μ—†λ‹€.
  • λ”°λΌμ„œ PIPE_ERRORλ₯Ό 읽도둝 λ³€κ²½λ§Œ 해두고, μˆ˜ν–‰λ˜μ§€ μ•Šμ€ 연산에 λŒ€ν•œ gotStatus() ν˜ΈμΆœμ€ INTERNAL: make lop piped operations process synchronouslyΒ #795 에 μΆ”κ°€ν•˜λ„λ‘ ν•œλ‹€.

@jhpark816 jhpark816 removed their request for review November 15, 2024 11:17
@jhpark816
Copy link
Collaborator

@uhm0311 λ¨Όμ € λ¦¬λ·°ν•˜λ©΄ λ©λ‹ˆλ‹€.

Copy link
Collaborator

@uhm0311 uhm0311 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1μ°¨ μ§ˆλ¬Έμž…λ‹ˆλ‹€.

Copy link
Collaborator

@uhm0311 uhm0311 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2μ°¨, 리뷰 의견과 μ§ˆλ¬Έμž…λ‹ˆλ‹€.

@oliviarla oliviarla force-pushed the pipeend branch 2 times, most recently from faec8fa to 20cf3d9 Compare November 19, 2024 06:48
@oliviarla oliviarla self-assigned this Nov 19, 2024
Copy link
Collaborator

@uhm0311 uhm0311 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3μ°¨ 리뷰 μ˜κ²¬μž…λ‹ˆλ‹€.

@jhpark816
Copy link
Collaborator

@oliviarla @uhm0311
CompositeException의 printCallStack() κ΄€λ ¨ PR은 merge λ˜μ—ˆμŠ΅λ‹ˆλ‹€.

@oliviarla oliviarla force-pushed the pipeend branch 2 times, most recently from 5493230 to 466ff60 Compare November 26, 2024 10:28
@oliviarla
Copy link
Collaborator Author

@uhm0311 @jhpark816
ν˜„μž¬ END/PIPE_ERRORκ°€ μ˜€μ§€ μ•Šμ•„ μ˜ˆμ™Έκ°€ λ°œμƒν•œλ‹€λ©΄ μ•„λž˜μ™€ 같이 λ‘œκΉ…μ΄ λ˜λŠ”λ°μš”, 이 λ•Œ μ–΄λ–€ μ—λŸ¬κ°€ μ–΄λ–€ μ—°μ‚°μ—μ„œ λ‚¬λŠ”μ§€ ν™•μΈν•˜κΈ° μ–΄λ ΅λ‹€λŠ” 단점이 있긴 ν•©λ‹ˆλ‹€λ§Œ, 별닀λ₯Έ ν•΄κ²° λ°©μ•ˆμ€ μ—†λŠ” 것 κ°™μŠ΅λ‹ˆλ‹€. (pipe operationμ—μ„œ λ‚΄λΆ€μ μœΌλ‘œ μ‚¬μš©ν•˜κ³  μžˆλŠ” index 정보λ₯Ό 같이 λ‘œκΉ…ν•˜λ©΄ μ–΄λ–¨κΉŒ μƒκ°ν•΄λ³΄μ•˜λŠ”λ°, λ©”μ„œλ“œ 인자 등을 둜그둜 μ•ŒκΈ° μ–΄λ €μš°λ―€λ‘œ indexλ₯Ό μ•Œλ”λΌλ„ μ“Έλͺ¨μ—†λŠ” 정보가 될 것 κ°™μŠ΅λ‹ˆλ‹€.)

OperationException: SERVER: Multiple exceptions (3) reported: SERVER_ERROR out of memory @ testnode 0.0.0.0/0.0.0.0:11211, CLIENT_ERROR too large value @ testnode 0.0.0.0/0.0.0.0:11211, END|PIPE_ERROR not received

@oliviarla oliviarla force-pushed the pipeend branch 2 times, most recently from 00d28cd to 63fac49 Compare November 26, 2024 10:53
@oliviarla oliviarla requested a review from uhm0311 November 28, 2024 01:30
Copy link
Collaborator

@uhm0311 uhm0311 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

4μ°¨ 리뷰 μ˜κ²¬μž…λ‹ˆλ‹€.

@oliviarla oliviarla force-pushed the pipeend branch 2 times, most recently from a752f86 to 7bf52e7 Compare November 28, 2024 03:09
uhm0311
uhm0311 previously approved these changes Nov 28, 2024
@oliviarla oliviarla requested a review from jhpark816 November 28, 2024 08:23
@oliviarla
Copy link
Collaborator Author

@jhpark816 리뷰 λΆ€νƒλ“œλ¦½λ‹ˆλ‹€.

@jhpark816
Copy link
Collaborator

@oliviarla 리뷰 진행 μ€‘μž…λ‹ˆλ‹€.

Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

일뢀 리뷰

Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 μ™„λ£Œ,
λ³΅μž‘ν•œ μ΄μŠˆλ„€μš”.

@oliviarla
Copy link
Collaborator Author

oliviarla commented Dec 9, 2024

@jhpark816 @uhm0311
ν˜„μž¬ μƒν™©μ—μ„œ μˆ˜μ •ν•  뢀뢄을 λŒ€λž΅ μ •λ¦¬ν•΄λ΄€μŠ΅λ‹ˆλ‹€. μ½”λ“œλ‘œ μž‘μ„±ν•˜κΈ° 전에 κ²€ν†  λ¨Όμ € λΆ€νƒλ“œλ¦½λ‹ˆλ‹€.

  • OperationImpl#readFromBuffer의 둜직 λ³€κ²½
    • isPipeOperation()κ°€ true이고 첫 번째 응닡이 RESPONSE <count> 라면, μ΄ν›„μ—λŠ” classifyError()λ₯Ό κ±°μΉ˜μ§€ μ•Šκ³  handleLine()μ—μ„œ
    • CLIENT_ERROR, SERVER_ERROR, ERRORλ₯Ό 읽고 gotStatus 콜백 λ©”μ„œλ“œμ— λ„˜κΈ΄λ‹€.
  • handleLine(String line)의 둜직 λ³€κ²½
    • ENDκ°€ 왔을 λ•Œμ—λŠ” Operation COMPLETE μƒνƒœλ‘œ λ³€κ²½
    • PIPE_ERRORκ°€ 왔을 λ•Œμ—λŠ” Operation COMPLETE μƒνƒœλ‘œ λ³€κ²½ν•˜κ³ , operation에 μ˜ˆμ™Έλ₯Ό μΆ”κ°€ν•΄ reconnectλ˜λ„λ‘ ν•˜κ³ , hasErroredκ°€ trueλ₯Ό λ°˜ν™˜ν•˜λ„λ‘ ν•œλ‹€.
  • getFailedResult() λ©”μ„œλ“œλ₯Ό pipe κ΄€λ ¨ Future에 μΆ”κ°€
    • μ—λŸ¬μ™€ 상관없이 failedResult κ²°κ³Όλ₯Ό μ‘°νšŒν•  수 μžˆλ„λ‘ ν•œλ‹€.

@jhpark816
Copy link
Collaborator

@oliviarla

isPipeOperation()κ°€ true이면 classifyError()λ₯Ό κ±°μΉ˜μ§€ μ•Šκ³  handleLine()μ—μ„œ CLIENT_ERROR, SERVER_ERROR, ERRORλ₯Ό gotStatus 콜백 λ©”μ„œλ“œμ— λ„˜κΈ΄λ‹€.

  • pipe μš”μ²­μ— λŒ€ν•΄ ERROR 등이 λ°”λ‘œ λ‚˜μ˜¬ 수 있으며, 이 κ²½μš°λŠ” classifyError()λ₯Ό μˆ˜ν–‰ν•΄μ•Ό ν•©λ‹ˆλ‹€.
  • pipe μš”μ²­μ— λŒ€ν•΄ RESPONSE <count>\r\n 응닡이 온 μ΄ν›„μ—λŠ” classifyError()λ₯Ό κ±°μΉ˜μ§€ μ•Šμ•„μ•Ό ν•©λ‹ˆλ‹€.

ENDκ°€ 였면 λ¦¬λ‹€μ΄λ ‰νŠΈν•œ ν›„ Operation COMPLETE μƒνƒœλ‘œ λ³€κ²½
PIPE_ERRORκ°€ 였면 λ¦¬λ‹€μ΄λ ‰νŠΈν•œ ν›„ Operation COMPLETE μƒνƒœλ‘œ λ³€κ²½

λ¦¬λ‹€μ΄λ ‰νŠΈν•œλ‹€λŠ” 것이 μ–΄λ–€ μ˜λ―ΈμΈκ°€μš”?

future.get()의 둜직 λ³€κ²½
hasErrored 일 λ•Œμ—λŠ” exception을 λ˜μ§€μ§€ μ•ŠλŠ”λ‹€.

resultκ°€ μžˆλŠ” 지에 따라 νŒλ‹¨ν•΄μ•Ό ν•  지 ? μ •λ„μ˜ λŠλ‚Œμ΄ λ“­λ‹ˆλ‹€.

Copy link
Collaborator

@uhm0311 uhm0311 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

λ³€κ²½ 내역을 λ‹€ 따라가지 λͺ»ν•΄μ„œ 질문 λ‚¨κΉλ‹ˆλ‹€.

@oliviarla oliviarla changed the title INTERNAL: read while END/PIPE_ERROR received in the pipe operation INTERNAL: read while PIPE_ERROR received in the pipe operation Dec 27, 2024
@oliviarla
Copy link
Collaborator Author

@uhm0311 리뷰 μ™„λ£Œλ˜μ…¨μ„κΉŒμš”?

@uhm0311
Copy link
Collaborator

uhm0311 commented Jan 2, 2025

@oliviarla

진행 μ€‘μž…λ‹ˆλ‹€.

uhm0311
uhm0311 previously approved these changes Jan 2, 2025
@oliviarla oliviarla requested a review from jhpark816 January 3, 2025 02:03
Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 μ™„λ£Œ

Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 μ™„λ£Œ

protected void handleError(OperationErrorType eType, String line) throws IOException {
getLogger().error("Error: %s by %s", line, this);
exception = createException(eType, line);
if (count == 0 && index == 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CollectionPipedInsertOperationImpl ν΄λž˜μŠ€μ—μ„œ
λͺ…μ‹œμ μœΌλ‘œ indexλ₯Ό 0으둜 μ΄ˆκΈ°ν™”ν•΄ λ‘λŠ” κ²ƒμ²˜λŸΌ
λͺ…μ‹œμ μœΌλ‘œ count도 0으둜 μ΄ˆκΈ°ν™”ν•΄ 두면 μ’‹κ² μŠ΅λ‹ˆλ‹€.

그리고, count == 0 쑰건만 μžˆμ–΄μ•Ό ν•˜κ³ , index == 0 쑰건은 λΉ μ Έμ•Ό ν•©λ‹ˆλ‹€.
μ•„λž˜ κ²½μš°μ— index == 0이기 λ•Œλ¬Έμž…λ‹ˆλ‹€.

RESPONSE 1
CLIENT_ERROR ...
PIPE_ERROR

그리고, switchover와 redirectν•˜λŠ” 경우λ₯Ό κ³ λ €ν•œλ‹€λ©΄,
count == 0 쑰건만 있으면 μΆ©λΆ„ν•œ 지 κ°„λ‹¨νžˆ μ„€λͺ…ν•΄ μ£Όμ„Έμš”.

Copy link
Collaborator Author

@oliviarla oliviarla Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switchover / redirectν•˜λŠ” 경우 count 값이 μ΄ˆκΈ°ν™”λ˜λ„λ‘ μˆ˜μ •ν–ˆμŠ΅λ‹ˆλ‹€. (count 값은 ν˜„μž¬ μ‚¬μš©ν•˜κ³  μžˆμ§€ μ•ŠκΈ° λ•Œλ¬Έμ— μ΅œμ‹  RESPONSE <count> 정보λ₯Ό μ €μž₯ν•΄λ‘λŠ” μš©λ„λ‘œλ§Œ μ‚¬μš©ν•΄λ„ λ©λ‹ˆλ‹€.)
이λ₯Ό 톡해 첫 번째 μ‘λ‹΅μœΌλ‘œ ERRORκ°€ 올 경우 λ°”λ‘œ μ˜ˆμ™Έλ₯Ό λ˜μ§€κ³ , RESPONSE 이후에 ERRORκ°€ 올 경우 PIPE_ERRORλ₯Ό 읽도둝 ν•©λ‹ˆλ‹€.

@oliviarla
Copy link
Collaborator Author

리뷰 λͺ¨λ‘ λ°˜μ˜ν–ˆμŠ΅λ‹ˆλ‹€.

uhm0311
uhm0311 previously approved these changes Jan 8, 2025
Copy link
Collaborator

@jhpark816 jhpark816 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

리뷰 μ™„λ£Œ

@@ -36,6 +36,8 @@
import net.spy.memcached.ops.OperationType;
import net.spy.memcached.ops.StatusCode;

import static net.spy.memcached.ops.OperationErrorType.CLIENT;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

μ§ˆλ¬Έμž…λ‹ˆλ‹€.

  • import staticμ—μ„œ static은 μ–΄λ–€ μ˜λ―Έκ°€ λ˜λ‚˜μš”?
  • κΈ°μ‘΄ μ½”λ“œμ—μ„œλ„ CLIENTκ°€ μ‚¬μš©λ˜κ³  μžˆμŠ΅λ‹ˆλ‹€. 이미 κ΄€λ ¨λœ importκ°€ μ‘΄μž¬ν•˜μ§€ μ•ŠλŠ” 지 ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static importλ₯Ό ν•˜κ²Œ 되면 ν•΄λ‹Ή λ³€μˆ˜λ‚˜ μƒμˆ˜λ₯Ό μ‚¬μš©ν•  λ•Œ μ •μ˜λœ 클래슀 이름을 λͺ…μ‹œν•˜μ§€ μ•Šκ³  μ‚¬μš©ν•  수 μžˆμŠ΅λ‹ˆλ‹€.
예λ₯Ό λ“€μ–΄ 일반 import μ‹œμ—λŠ” OperationErrorType.CLIENT둜 μ‚¬μš©ν•΄μ•Ό ν•˜μ§€λ§Œ, static import μ‹œμ—λŠ” CLIENT둜 μ‚¬μš©ν•  수 μžˆλŠ”λ°, κΈ°μ‘΄μ—λŠ” switch문을 μ‚¬μš©ν•΄μ„œ static importν•  ν•„μš”κ°€ μ—†μ—ˆμŠ΅λ‹ˆλ‹€.

λ‹€λ§Œ CLIENT만 μ‚¬μš©ν•˜λŠ” 것보단 OperationErrorType.CLIENT μ‚¬μš©ν•˜λŠ” 것이 더 이해가 μ‰¬μšΈ 것 κ°™μ•„ static importλ₯Ό μ œκ±°ν–ˆμŠ΅λ‹ˆλ‹€.

@@ -99,9 +102,11 @@ assert getState() == OperationState.READING
if (hasSwitchedOver(line)) {
this.insert.setNextOpIndex(index);
prepareSwitchover(line);
count = 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switchover 경우λ₯Ό 예둜 λ“€λ©΄,
이미 일뢀가 μˆ˜ν–‰λœ μƒνƒœμ—μ„œ λ‚˜λ¨Έμ§€λ₯Ό switchoverν•œ 후에 μˆ˜ν–‰ν•˜λ©΄,
κ·Έ μ΄ν›„μ˜ κ²°κ³ΌλŠ” μ›μ†Œ 결과둜 μ²˜λ¦¬ν•΄μ•Ό ν•  것 κ°™μŠ΅λ‹ˆλ‹€.

NOT_MY_KEY κ²½μš°λ„ ν•¨κ»˜ κ²€ν†  λ°”λžλ‹ˆλ‹€.

Copy link
Collaborator Author

@oliviarla oliviarla Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switchover, redirect 둜 인해 μƒˆλ‘œμš΄ 별도 μš”μ²­μ„ 보낸 ν›„ 응닡을 λ°›μ•˜μ„ λ•Œ, RESPONSEκ°€ 올 μˆ˜λ„ 있고 ν•˜λ‚˜μ˜ ERROR 라인이 올 μˆ˜λ„ μžˆμŠ΅λ‹ˆλ‹€.
이 λ•Œ RESPONSEκ°€ 왔을 λ•Œμ—λ§Œ μ˜ˆμ™Έλ₯Ό λ˜μ§€μ§€ μ•Šλ„λ‘ ν•˜μ—¬ PIPE_ERRORκΉŒμ§€ 읽도둝 ν•©λ‹ˆλ‹€.

@oliviarla
Copy link
Collaborator Author

@jhpark816
count λŒ€μ‹  readUntilLastLine ν”Œλž˜κ·Έλ₯Ό 두도둝 λ³€κ²½ν•˜κ³  주석도 μΆ”κ°€ν–ˆμŠ΅λ‹ˆλ‹€.

@jhpark816 jhpark816 merged commit fa44194 into naver:develop Jan 9, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants