Skip to content

Commit

Permalink
update howto for contributing
Browse files Browse the repository at this point in the history
  • Loading branch information
@mampfes
mampfes committed Apr 6, 2023
1 parent b0b3d4e commit 154a197
Show file tree
Hide file tree
Showing 5 changed files with 346 additions and 213 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -560,7 +560,7 @@ There are several ways of contributing to this project, they include:
- Helping answer/fix any issues raised
- Join in with the Home Assistant Community discussion

For further details see [contribution](/doc/contributing.md) guidelines, or take a look at our [online](/doc/online.md) mentions.
For further details see [contributing](/doc/contributing.md) guidelines, or take a look at our [online](/doc/online.md) mentions.

<!--
# Development Roadmap
Expand Down
2 changes: 1 addition & 1 deletion custom_components/waste_collection_schedule/waste_collection_schedule/test/test_sources.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ def main():
# run through all .yaml files for ICS source
module = importlib.import_module("waste_collection_schedule.source.ics")
for f in sorted(yaml_files):
print(f"Testing ICS {f.name}")
print(f"Testing ICS {f.stem}")
with open(f) as stream:
# read yaml file
data = yaml.safe_load(stream)
Expand Down
217 changes: 6 additions & 211 deletions doc/contributing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,222 +45,17 @@ git branch adding_abc_com

For more info on forking/cloning a repository, see GitHub's [fork-a-repo](https://docs.github.com/en/get-started/quickstart/fork-a-repo) document.

### Files Required For A New Service Provider
### The 2 Ways of adding support for a new Service Provider

The following files need to be provided to support a new service provider:
There are 2 ways to add support for a new service provider:

- A python `source script` that retrieves the collection schedule information, formats it appropriately, and has test cases that can be used to confirm functionality.
- A `source markdown (.md)` file that describes how to configure the new source and sensor, with examples.
- An updated `README.md` file containing details of the new service provider.
- An updated `info.md` file containing details of the new service provider.
1. [Via the generic ICS source](contributing_ics.md)

The framework contains a [test script](#test-the-new-source-file) that can be used to confirm source scripts are retrieving and returning correctly formatted waste collection schedules.
This is the preferred way of adding support for a new service provider, but only works if the service providers offers a so called "ical / webcal subscription" or at least a static link which doesn't change over time to an ICS file.

### Python Source Script
2. [Dedicated source](contributing_source.md)

Create a new file in the `custom_components/waste_collection_schedule/waste_collection_schedule/source` folder. The file name should be the url of your service provider in lower case, for example `abc_com.py` for `https://www.abc.com`.

The script should have the following general structure

```py
import datetime
from waste_collection_schedule import Collection

TITLE = "My Council" # Title will show up in README.md and info.md
DESCRIPTION = "Source script for abc.com" # Describe your source
URL = "https://abc.com" # Insert url to service homepage. URL will show up in README.md and info.md
TEST_CASES = { # Insert arguments for test cases to be used by test_sources.py script
"TestName1": {"arg1": 100, "arg2": "street"},
"TestName2": {"arg1": 200, "arg2": "road"},
"TestName3": {"arg1": 300, "arg2": "lane"}
}

API_URL = "https://abc.com/search/"
ICON_MAP = { # Optional: Dict of waste types and suitable mdi icons
"DOMESTIC": "mdi:trash-can",
"RECYCLE": "mdi:recycle",
"ORGANIC": "mdi:leaf",
}


class Source:
def __init__(self, arg1, arg2): # argX correspond to the args dict in the source configuration
self._arg1 = arg1
self._arg2 = arg2

def fetch(self):

# replace this comment with
# api calls or web scraping required
# to capture waste collection schedules
# and extract date and waste type details

entries = [] # List that holds collection schedule

entries.append(
Collection(
date = datetime.datetime(2020, 4, 11), # Collection date
t = "Waste Type", # Collection type
icon = ICON_MAP.get("Waste Type"), # Collection icon
)
)

return entries
```

Filtering of data for waste types or time periods is a functionality of the framework and should not be done by the source script. Therefore:

- A source script should return all data for all available waste types.
- A source script should **not** provide options to limit the returned waste types.
- A source script should return all data for the entire time period available (including past dates if they are returned).
- A source script should **not** provide a configuration option to limit the requested time frame.

### Service Provider Markdown File

Create a new markdown file in the `custom_components/waste_collection_schedule/doc/source` folder. The file name should be the url of your service provider in lower case, for example `abc_com.md` for `https://www.abc.com`.

The markdown file should have the following general structure:

1. A description of how the source should be configured.
2. A description of the arguments required, the type, and whether they are optional/mandatory.
3. A working example (usually one of the test cases from the `.py` file)

For example:

**Configuration via configuration.yaml**

```yaml
waste_collection_schedule:
sources:
- name: abc_com
args:
uprn: UNIQUE_PROPERTY_REFERENCE_NUMBER
```
**Configuration Variables**
**uprn** _(string) (required)_ : The unique 12-digit identifier for your property
Example:
```yaml
waste_collection_schedule:
sources:
- name: abc_com
args:
uprn: "e3850cac2d5b"
```
Note: Your uprn can be found on invoices from your service provider
### Update Links in README.md and info.md
The `README.md` file in the top level folder contains a list of supported service providers.

The `info.md` is rendered in the HACS user interface within Home Assistant and gives potential users a summary of what the component does, and the service providers supported.

The links in both files can be updated automatically using the script `update_docu_links.py` in the top-level directory:

```bash
./update_docu_links.py
```

The script iterates through all source files and extracts some meta information like title and url. It is therefore important to set the attributes in the source file correctly. By default, the country classification is derived from the file name. If this doesn't match, the country code can be overwritten with the attribute `COUNTRY`.

| Attribute | Type | Description |
|-|-|-|
| TITLE | String | Title of the source. Used as link title in README.md and info.md. |
| URL | String | Service provider homepage URL. The idea is to help users to identify their service provider if they search for an URL instead of a service provider name. The abbreviated domain name is therefore displayed next to the source title in README.md. |
| COUNTRY | String | [Optional] Overwrite default country code which is derived from source file name. |
| EXTRA_INFO | List of Dicts or Callable | [Optional] Used to add extra links in README.md and info.md if the source supports multiple service providers at the same time. The following keys can be added to the dict: `title`, `url`, `country`. In case a key is missing, the corresponding value from the attributes above will be used instead. |

Examples:

```python
# Standard case: Source supports one service provider
TITLE = "ART Trier"
URL = "https://www.art-trier.de"
# Special case: Overwrite country code
TITLE = "RecycleSmart"
URL = "https://www.recyclesmart.com/"
COUNTRY = "au"
# Special case: Source supports multiple service provider which should be added to README.md and info.md
TITLE = "FCC Environment"
URL = "https://fccenvironment.co.uk"
EXTRA_INFO = [
{
"title": "Harborough District Council",
"url": "https://harborough.gov.uk"
},
{
"title": "South Hams District Council",
"url": "https://southhams.gov.uk/"
},
]
# Special case: Same as before, but EXTRA_INFO created by a function from existing data
TITLE = "Bürgerportal"
URL = "https://www.c-trace.de"
def EXTRA_INFO():
return [ { "title": s["title"], "url": s["url"] } for s in SERVICE_MAP ]
```

### Test The New Source File

Debugging a source script within Home Assistant is not recommended. Home Assistant's start-up process is too slow for fast debugging cycles. To help with debugging/troubleshooting, the Waste Collection Schedule framework contains a command line script that can be used to test source scripts. The script iterates through the `test cases` defined in the source script passing each set of arguments to the source script and prints the results.

The script supports the following options:

| Option | Argument | Description |
|--------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------|
| `-s` | SOURCE | [Source name](https://github.com/mampfes/hacs_waste_collection_schedule#source-configuration-variables) (source file name without ending `.py`) |
| `-l` | - | List all found dates. |
| `-i` | - | Add icon name to output. Only effective together with `-l`. |
| `-t` | - | Show extended exception info and stack trace. |

For debugging purposes of a single source, it is recommended to use the `-s SOURCE` option. If used without any arguments provided, the script tests every script in the `custom_components/waste_collection_schedule/waste_collection_schedule/source` folder and prints the number of found entries for every test case.

To use it:

1. Navigate to the `custom_components/waste_collection_schedule/waste_collection_schedule/test/` directory
2. Confirm the `test_sources.py` script is present
3. Execute the test script. For example, testing the abfall_io.py source script would be:

```bash
test_sources.py -s abfall_io
```

4. Confirm the results returned match expectation. For example, testing abfall_io returns:

```text
Testing source abfall_io ...
found 285 entries for Waldenbuch
found 58 entries for Landshut
found 109 entries for Schoenmackers
found 3 entries for Freudenstadt
found 211 entries for Ludwigshafen am Rhein
found 119 entries for Traunstein
found 287 entries for Thalheim
```

5. To view individual date entries and assigned icons, use the `-i -l` arguments, for example:

```bash
test_sources.py -s richmondshire_gov_uk -i -l
Testing source richmondshire_gov_uk ...
found 53 entries for test 1
2023-01-02: 240L GREY RUBBISH BIN [mdi:trash-can]
2023-01-07: 55L RECYCLING BOX [mdi:recycle]
2023-01-13: 240L GREY RUBBISH BIN [mdi:trash-can]
2023-01-20: 55L RECYCLING BOX [mdi:recycle]
2023-01-27: 240L GREY RUBBISH BIN [mdi:trash-can]
...
2023-12-01: 240L GREY RUBBISH BIN [mdi:trash-can]
2023-12-08: 55L RECYCLING BOX [mdi:recycle]
2023-12-15: 240L GREY RUBBISH BIN [mdi:trash-can]
```
This is the fallback if the preferred way via the generic ICS source doesn't work.

### Sync Branch and Create A Pull Request

Expand Down
118 changes: 118 additions & 0 deletions doc/contributing_ics.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
<img src="/images/icon.png" alt="Waste Collection Schedule logo" title="Waste Collection Schedule" align="right" height="60" />

# Add support via generic ICS source

## Files required for a new Service Provider

The following files need to be provided to support a new service provider:

- A `yaml based ICS configuration` file that contains the name and the url of the supported service provider and has test cases that can be used to confirm functionality.
- A `source markdown (.md)` file that describes how to configure the new source and sensor, with examples. This file will be generated automatically out of the yaml file
- An updated `README.md` file containing details of the new service provider. This file will be automatically updated by a script.
- An updated `info.md` file containing details of the new service provider. This file will be automatically updated by a script.

The framework contains a [test script](#test-the-new-source-file) that can be used to confirm source scripts are retrieving and returning correctly formatted waste collection schedules.

## yaml based ICS configuration

Create a new file in the `/doc/ics/yaml` folder. The file name should be the url of your service provider in lower case, for example `abc_com.yaml` for `https://www.abc.com`.

The yaml file should have the following general structure

```yaml
title: TITLE
url: URL
country: COUNTRY
howto: HOWTO
test_cases: TEST_CASES
```
| Attribute | Type | Description |
|-|-|-|
| title | String | Title of the service provider. Used as link title in README.md and info.md. |
| url | String | Service provider homepage URL. The idea is to help users to identify their service provider if they search for an URL instead of a service provider name. The abbreviated domain name is therefore displayed next to the source title in README.md. |
| country | String | [Optional] Overwrite default country code which is derived from yaml file name. |
| howto | String | A multi-line string in markdown format which describes the steps to configure the ICS source. |
| test_cases | Dict | A dictionary with test-cases. The key of an entry represents the name of the test-case which will be displayed during testing. The item contains a dictionary of the source arguments. |
Example:
```yaml
title: Entsorgungsgesellschaft Görlitz-Löbau-Zittau
url: https://www.abfall-eglz.de
howto: |
- Goto <https://www.abfall-eglz.de/abfallkalender.html> and select your municipality.
- Right-click on `Entsorgungstermine als iCalendar herunterladen` and copy link address.
- Replace the `url` in the example configuration with this link.
test_cases:
Oppach:
url: "https://www.abfall-eglz.de/abfallkalender.html?ort=Oppach&ortsteil=Ort+Oppach&strasse=&ics=1"
split_at: " & "
```
## Service Provider Markdown File
The markdown file will be automatically generated from the information in the yaml file. Just call `update_docu_links.py` in the top-level directory. This will also update README.md and info.md automatically.

```bash
./update_docu_links.py
```

## Update Links in README.md and info.md

The `README.md` file in the top level folder contains a list of supported service providers.

The `info.md` is rendered in the HACS user interface within Home Assistant and gives potential users a summary of what the component does, and the service providers supported.

The links in both files can be updated automatically using the script `update_docu_links.py` in the top-level directory:

```bash
./update_docu_links.py
```

## Test the new ICS configuration

Debugging a source script within Home Assistant is not recommended. Home Assistant's start-up process is too slow for fast debugging cycles. To help with debugging/troubleshooting, the Waste Collection Schedule framework contains a command line script that can be used to test source scripts. The script iterates through the `test cases` defined in the source script passing each set of arguments to the source script and prints the results.

The script supports the following options:

| Option | Argument | Description |
|--------|----------|-|
| `-I` | - | Test all yaml files. |
| `-y` | YAML | yaml file name in folder `/doc/ics/yaml` without ending `.yaml` |
| `-l` | - | List all found dates. |
| `-i` | - | Add icon name to output. Only effective together with `-l`. |
| `-t` | - | Show extended exception info and stack trace. |

For debugging purposes of a single yaml configuration, it is recommended to use the `-y YAML` option. If used without any arguments provided, the script tests every script in the `custom_components/waste_collection_schedule/waste_collection_schedule/source` folder and prints the number of found entries for every test case.

To use it:

1. Navigate to the `/custom_components/waste_collection_schedule/waste_collection_schedule/test/` directory
2. Confirm the `test_sources.py` script is present
3. Execute the test script. For example, testing the abc_com.yaml configuration file would be:

```bash
test_sources.py -y koblenz_de
```

4. Confirm the results returned match expectation. For example, testing koblenz_de returns:

```text
Testing ICS koblenz_de
found 60 entries for Altstadt
```

5. To view individual date entries, use the `-l` arguments, for example:

```bash
test_sources.py -y koblenz_de -l
Testing ICS koblenz_de
found 60 entries for Altstadt
2023-04-19 : Altpapier
2023-05-10 : Altpapier
2023-06-01 : Altpapier
2023-06-21 : Altpapier
2023-07-12 : Altpapier
...
```
Loading

0 comments on commit 154a197

Please sign in to comment.