ED: Remove Datasets from search

[SpeckiJ]

I want to remove my dataset from the full text search (after i initially added it) because the paper this dataset is based on was retracted

[ridoo]

Opt-Out

[ridoo]

• Nachträglich aus dem Index nehmen
  • Abgleich mit Datenrepository
  • Neu trainieren oder unlearn?
• Idea for "end user": Können Ergebnisse mit ähnlichem "Kriterien-Index" verglichen werden?

[ridoo]

Two step approach

• Get Id of dataset (optionally in combination with search parameters)
• Delete Endpoint where the delete happens (by the data steward)

[simeonwetzel]

Hier sind diese endpoints implementiert:
https://github.com/52North/innovation-prize/blob/dev/search-app/server/app/server.py

# Clear complete index
@app.get("/clear_index")
async def clear_index(index_name: str):
    if index_name not in (indexes, 'geojson'):
        raise HTTPException(status_code=400, detail="Invalid index name")
    elif index_name == 'geojson':
       local_file_indexer._clear()
    else: 
        indexes[index_name]._clear()
    return {'message': 'Index cleared'}

# Get specific document from index
@app.get("/retrieve_with_id")
async def retrieve_with_id(index_name: str, _id: str):
    if index_name not in indexes:
        raise HTTPException(status_code=400, detail="Invalid index name")
    retrieved_id = indexes[index_name]._get_doc_by_id(_id)
    return {'id': retrieved_id}

# Delete specifc document from index
@app.get("/remove_doc_from_index")
async def remove_doc_from_index(index_name: str, _id: str):
    if index_name not in indexes:
        raise HTTPException(status_code=400, detail="Invalid index name")
    result = indexes[index_name]._delete_doc_from_index(_id)
    return result

[simeonwetzel]

Todo: Authentification mechanism for upsert/delete in index

[simeonwetzel]

Added an OAuth2-based authentification mechanism with api-key. The provider can specify api-keys in the config file and the user needs to add the key to the request header when requesting the API-endpoint. The authentification mechanism was added to all security-relevant endpoints

innovation-prize/search-app/server/app/server.py

Line 92 in 684126d

async def get_api_key(api_key_header: str = Security(api_key_header)):