Store the source to know from which apps the prices are created

[raphodn]

Story

• new Price.source field
• new Proof.source field
• store the User agent ? no the app_name URL param (see discussion below)
• Update the web app (send the app_name param) : feat(API): send the app_name param in every API call open-prices-frontend#588
• app_version : see Price/Proof source: store mobile app version #502
• new Location.source field

Why

To differentiate the apps that interact with the API and create the prices.

Related issues

Following discussion here : openfoodfacts/smooth-app#5195

[raphodn]

@monsieurtanuki @teolemon a few questions :

• it's just about storing the incoming User-Agent, right ?
• in Replicate the 'Add a single price' view from the Open Prices web app smooth-app#5195 you mention app_name, app_version... is it multiple fields, or one string with all that info ?
• could you give an example of the current UA sent by the mobile app ?

current PR 👇

// with Chrome
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36

// with Firefox
Open Prices Web App

any specific recommentation on the name to use ?

[teolemon]

I was more thinking about letting clients send arbitrary parameters as opposed to trying to be smart and getting the user agent.

[raphodn]

How is it done already ? I'd rather simply replicate your current behavior on the main app. But maybe you don't store it actually ?

[monsieurtanuki]

@raphodn This is currently the "user agent" we use in off-dart:

class UserAgent extends JsonObject {
  /// The name of your application (eg: smooth-app)
  final String name;

  /// The version of the application (1.0.0)
  final String? version;

  /// The system running the application (eg: Android+10)
  final String? system;

  /// The url of your application (eg: https://example.com)
  final String? url;

  /// Additional information about your application
  final String? comment;
// ...

And we use it approximately this way, adding parameters to GET for instance:

    map['app_name'] = OpenFoodAPIConfiguration.userAgent!.name;
    map['app_version'] = OpenFoodAPIConfiguration.userAgent!.version!;
    map['app_uuid'] = OpenFoodAPIConfiguration.uuid!;
    map['app_platform'] = OpenFoodAPIConfiguration.userAgent?.system ?? '';
    map['comment'] = OpenFoodAPIConfiguration.userAgent?.comment ?? '';

In short: we somehow add 5 parameters to GET/POST queries in order to know more about the caller.

• do you need all these 5 fields? I don't know.
• are these fields coming from "user agents"? Not really, we populate them with whatever is relevant.
• what could make sense, as is done with other tools, is to populate at least a single "app_name", e.g. populated with a distinct unique value like com.mybestprices.app.

[raphodn]

https://openfoodfacts.github.io/openfoodfacts-server/api/#authentication

So the OpenFoodAPIConfiguration groups everything in one string afterwards ? Or it sends multiple headers ?

[teolemon]

The documentation is outdated, we now have a structured way to do it.

[teolemon]

https://github.com/openfoodfacts/openfoodfacts-server/blob/2dffe277bc2f7955ca40a315aaac553561aeb6bc/lib/ProductOpener/Products.pm#L1410

[teolemon]

I've created a doc PR: openfoodfacts/openfoodfacts-server#10368

[raphodn]

In short: we somehow add 5 parameters to GET/POST queries in order to know more about the caller.

Sorry to insist 😅 Can you give a concrete example of a GET & POST request ? To see if these parameters are in the url, or in the body, or in the headers...

And what other (web) apps can I inspect that pass these info to the backend ? I just checked hunger games and I don't see anything for instance

[monsieurtanuki]

@raphodn I'm French and I'm an engineer, therefore I have to say: it depends.

AFAIR we can use our user agent in the header (rather for http server stat purpose), and as query parameters (for more refined stat purpose). Always in the header, and sometimes as query parameters too.

As I said in a previous message our user agent is split in 1 mandatory field (app_name) and 4 optional fields (app_version, app_uuid, app_platform, comment)

• We need the user agent variable to be populated
• Each populated user agent field is added to the query parameters - if the user agent is to be added to query parameters (always the case in our GET calls for instance)
• The user agent is always added to the header, as a concatenation of all populated user agent fields (separated by a hyphen)

For instance, for our "get product by barcode" query:

• https://world.openfoodfacts.org/api/v3/product/0028400047685/
  • ?lc=en
  • &tags_lc=en
  • &app_name=off-dart+integration+tests
• header:
  • Accept: application/json
  • User-Agent: - off-dart integration tests
  • From: grumpf

[raphodn]

Thanks @monsieurtanuki for the detailed answer.

So I'd rather not store the User Agent then (and leave it for http server stats), but look at the app_name URL param 👍

I'll implement this in the backend & web frontend !

[raphodn]

@monsieurtanuki ok it's in prod. simply passing a ?app_name=XXX will do it.

I did the changes in the frontend web app and it's in prod as well : openfoodfacts/open-prices-frontend#588

[monsieurtanuki]

Hi @raphodn!

I've just tested in .net env and added the app_name URI parameter in both "upload proof" and "create price" methods.
OK I added the parameter.

But where am I supposed to see the source field in the results? Couldn't find it in "price" or "proof"...
And even then, the naming wouldn't be very wise, as we also have a source field in "product" where we expect values like "off" or "obf".

[raphodn]

The field is for internal stats only for now :)
To know from which sources the prices & proofs are created.

Not sure if there is any value in returning that data in the JSON responses.

You're right the naming is similar to the flavor, i'll see if I rename it or leave it as is for the time being..

[monsieurtanuki]

@raphodn I would suggest naming it app_name ;)

[raphodn]

Ideally i would rename source to flavor in OFF 😅

[monsieurtanuki]

Ideally i would rename source to flavor in OFF 😅

If so, please create a specific issue for that in https://github.com/openfoodfacts/openfoodfacts-dart