Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/Schwarzam/lsdb-server into …
Browse files Browse the repository at this point in the history
…main
  • Loading branch information
Schwarzam committed May 25, 2024
2 parents 7f08156 + b7fb985 commit e463b09
Show file tree
Hide file tree
Showing 3 changed files with 2,249 additions and 330 deletions.
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,6 @@ Cargo.lock
**/*.rs.bk

# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb
*.pdb

.DS_Store
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,17 @@ A lightweight, fast and easy to use server for the [lsdb server test branch](htt

---

#### TODO list:

- [ ] Accept lower case column names
- [ ] Get columns by index
- [ ] Exclude columns by name
- [ ] !! Fix `_hipscat_index` parquet schema !!

### Configuring the server

`lsdb-server` works with [lsdb server test branch](https://github.com/Schwarzam/lsdb/tree/server-test) if you have the hips partitioned catalogs in your server.

Generate the hips using [hipscat-import](https://lsdb.readthedocs.io/en/latest/tutorials/import_catalogs.html).

---
Expand Down Expand Up @@ -59,4 +68,16 @@ server {

- The server only works with lsdb while using dtype_backend = "numpy_nullable"

This is because polars writes strings as LargeStrings or LargeUtf8, which causes issues with the metadata read in lsdb.

### Parameters

- `columns` : The columns to return in the response. Default is all columns. Default is all columns.
- `exclude_cols`: The columns to exclude from the response. Default is no columns.
- `filters`: A list of filters to apply ["r_auto < 18", ...]. Default is no filters.

### Caveats

- The server only works with lsdb while using dtype_backend = "numpy_nullable"

This is because polars writes strings as LargeStrings or LargeUtf8, which causes issues with the metadata read in lsdb.
Loading

0 comments on commit e463b09

Please sign in to comment.