Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: check eth events indexed in range #12728

Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@

- Make `EthTraceFilter` / `trace_filter` skip null rounds instead of erroring. ([filecoin-project/lotus#12702](https://github.com/filecoin-project/lotus/pull/12702))
- Event APIs (`GetActorEventsRaw`, `SubscribeActorEventsRaw`, `eth_getLogs`, `eth_newFilter`, etc.) will now return an error when a request matches more than `MaxFilterResults` (default: 10,000) rather than silently truncating the results. Also apply an internal event matcher for `eth_getLogs` (etc.) to avoid builtin actor events on database query so as not to include them in `MaxFilterResults` calculation. ([filecoin-project/lotus#12671](https://github.com/filecoin-project/lotus/pull/12671))
- `ChainIndexer#GetMsgInfo` returns an `ErrNotFound` when there are no rows. ([filecoin-project/lotus#12680](https://github.com/filecoin-project/lotus/pull/12680))
- `ChainIndexer#GetMsgInfo` returns an `ErrNotFound` when there are no rows. ([filecoin-project/lotus#12680](https://github.com/filecoin-project/lotus/pull/12680))\
- Return error if logs or events within range are not indexed. ([filecoin-project/lotus#12728](https://github.com/filecoin-project/lotus/pull/12728))

## New Features

Expand Down
35 changes: 35 additions & 0 deletions chain/index/events.go
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,8 @@ func (si *SqliteIndexer) checkTipsetIndexedStatus(ctx context.Context, f *EventF

return xerrors.Errorf("failed to get tipset key cid by height: %w", err)
}
case f.MinHeight >= 0 && f.MaxHeight >= 0 && f.MinHeight != f.MaxHeight:
return si.checkRangeIndexedStatus(ctx, f)
default:
// This function distinguishes between two scenarios:
// 1. Missing events: The requested tipset is not present in the Index (an error condition).
Expand All @@ -280,6 +282,39 @@ func (si *SqliteIndexer) checkTipsetIndexedStatus(ctx context.Context, f *EventF
return ErrNotFound // Tipset is not indexed
}

func (si *SqliteIndexer) checkRangeIndexedStatus(ctx context.Context, f *EventFilter) error {
startCid, err := si.getTipsetKeyCidByHeight(ctx, f.MinHeight)
if err != nil {
if errors.Is(err, ErrNotFound) {
// Null round for the start of the range is acceptable
return nil
}
return xerrors.Errorf("failed to get tipset key cid for start height: %w", err)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the ErrNotFound case, I'd like it to increment by 1 until it finds one <= MaxHeight that's not a null round.

In the special case where there are only null rounds between MinHeight and MaxHeight, pretend that it's indexed and don't return an error, they'll just get no results.

So you'll need a loop here, and a local copy of f.MinHeight to increment.


endCid, err := si.getTipsetKeyCidByHeight(ctx, f.MaxHeight)
if err != nil {
if errors.Is(err, ErrNotFound) {
// Check if the range ends in the future
head := si.cs.GetHeaviestTipSet()
if head == nil || f.MaxHeight > head.Height() {
return xerrors.Errorf("range end is in the future: %w", err)
}
return ErrNotFound // End is unindexed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're back in the null round case, I think that we should walk backward to find the first non-null lower than MaxHeight. Basically the same as the approach outlined above for MinHeight but in the other direction. You may find a way to abstract the algorithm to make it a reusable function, or it may be easier to just write it all out.

Basically: let's contract the range if the edges contain null rounds, walk inward until we find a min and max that are not null rounds. Be careful not to iterate beyond the other bound (don't increment from Min beyond the Max value, and don't decrement from Max beyond the Min value). The special case is when there are only null rounds between them, don't error, just return. There's another special case of contracting to min==max, you could easily do an if block with startCid.Equals(endCid) around the second isTipsetIndexed call to deal with that (although it's not critical since it'd just mean two queries for the same thing if it wasn't handled).

}
return xerrors.Errorf("failed to get tipset key cid for end height: %w", err)
}

if exists, err := si.isTipsetIndexed(ctx, startCid); err != nil || !exists {
return xerrors.Errorf("start tipset not indexed: %w", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return xerrors.Errorf("start tipset not indexed: %w", err)
return xerrors.Errorf("filter start tipset is not indexed: %w", err)

}
if exists, err := si.isTipsetIndexed(ctx, endCid); err != nil || !exists {
return xerrors.Errorf("end tipset not indexed: %w", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return xerrors.Errorf("end tipset not indexed: %w", err)
return xerrors.Errorf("filter end tipset is not indexed: %w", err)

}

return nil
}

// getTipsetKeyCidByHeight retrieves the tipset key CID for a given height.
func (si *SqliteIndexer) getTipsetKeyCidByHeight(ctx context.Context, height abi.ChainEpoch) ([]byte, error) {
ts, err := si.cs.GetTipsetByHeight(ctx, height, nil, false)
Expand Down
Loading