Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CAFDI-7 - Create Multi TLD Gravity Model MVP #7

Merged
merged 95 commits into from
Jan 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
69a77a8
Added module and comments for Multi area GM
wsp-mbuckley Sep 7, 2023
5eeac31
Added module and comments for Multi area GM
wsp-mbuckley Sep 7, 2023
1e25262
Merge branch 'multi_tld_mvp' of https://github.com/Transport-for-the-…
wsp-mbuckley Nov 10, 2023
3e81f07
Moved gravity and jacobian methods to single area
wsp-mbuckley Nov 10, 2023
61075b3
Changed furness to work for non-square matrices
isaac-tfn Nov 24, 2023
d2bae20
Rough structure of multi-tld
isaac-tfn Nov 24, 2023
1b5037c
Seems to work roughly
isaac-tfn Nov 27, 2023
fe87b7b
Pushing changes to work on laptop because no internet
isaac-tfn Nov 28, 2023
d4b6934
Crude convergence check
isaac-tfn Nov 28, 2023
9eef536
Ready to mp
isaac-tfn Nov 28, 2023
5180db5
Warning raised for zeros in the seed matrix
isaac-tfn Nov 29, 2023
1cac8d7
New utils module
isaac-tfn Nov 29, 2023
24fbdff
Written new jacobian to account for all mu and sigma. Untried.
isaac-tfn Nov 30, 2023
6d1f15c
New gravity written
isaac-tfn Nov 30, 2023
ca2e3b9
Triply constrained furness is commented out
isaac-tfn Dec 1, 2023
fdf7b84
Updated methods to perform jacobian all at once
isaac-tfn Dec 1, 2023
a7c5b3e
Added todo to add multi-area type infilling
isaac-tfn Dec 1, 2023
53e3ee7
Param guesses were duplicated
isaac-tfn Dec 1, 2023
05012a2
Some more tidying
isaac-tfn Dec 1, 2023
06baaf5
Updated documentation
isaac-tfn Dec 5, 2023
471fdb2
Refactor
isaac-tfn Dec 5, 2023
efaaab7
File for testing
isaac-tfn Dec 5, 2023
97f7f4b
Some refactoring - multi runs ok
isaac-tfn Dec 5, 2023
0e73e51
Changed jacobian to work with different furnessing tolerances in grav…
isaac-tfn Dec 6, 2023
9a0d422
Black
isaac-tfn Dec 6, 2023
e98f260
Ready to run
isaac-tfn Dec 6, 2023
e86b8c4
Added plot method to results
isaac-tfn Dec 7, 2023
38e76b4
Written multi-tld tests
isaac-tfn Dec 11, 2023
26f5e50
Refactored
isaac-tfn Dec 11, 2023
d3b3cb3
Fixed tests for new format
isaac-tfn Dec 12, 2023
3278725
Test furness/no furness in jacobian
isaac-tfn Dec 12, 2023
84e4cf1
Black
isaac-tfn Dec 12, 2023
ad727d1
Simplify run method
isaac-tfn Dec 12, 2023
78c9357
Added matplotlib as dependency
isaac-tfn Dec 13, 2023
10feeeb
Fixed absolute path in tests
isaac-tfn Dec 13, 2023
35c3ec6
Fixed some linting errors
isaac-tfn Dec 13, 2023
534481b
mypy checks
isaac-tfn Dec 13, 2023
f8245d0
black
isaac-tfn Dec 13, 2023
774fa27
Update src/caf/distribute/furness.py
isaac-tfn Jan 4, 2024
8e62eea
Update src/caf/distribute/gravity_model/multi_area.py
isaac-tfn Jan 4, 2024
f41d596
Update src/caf/distribute/gravity_model/multi_area.py
isaac-tfn Jan 4, 2024
79b2e7b
Update src/caf/distribute/utils.py
isaac-tfn Jan 4, 2024
f8896c1
Update src/caf/distribute/gravity_model/single_area.py
isaac-tfn Jan 4, 2024
97c44c8
Apply suggestions from code review
isaac-tfn Jan 4, 2024
7185f27
Comment fixes from review
isaac-tfn Jan 4, 2024
208b641
Changes from merging
isaac-tfn Jan 4, 2024
88389cd
Black + mypy
isaac-tfn Jan 4, 2024
afed201
Black
isaac-tfn Jan 4, 2024
1490af2
Altered sphinx build action
isaac-tfn Jan 4, 2024
ad4cd9a
Altered release.yml
isaac-tfn Jan 4, 2024
bbfbce6
Altered docs for build to work
isaac-tfn Jan 4, 2024
4108bcd
versioningit needed in docs req
isaac-tfn Jan 4, 2024
28da14f
Trying again. Sphinx builds successfully locally
isaac-tfn Jan 4, 2024
40244fb
changed readthedocs.yml
isaac-tfn Jan 4, 2024
161abfd
Trying again
isaac-tfn Jan 4, 2024
2e8e189
h
isaac-tfn Jan 4, 2024
e727a4e
Trying again
isaac-tfn Jan 4, 2024
b3f7b15
small typo
isaac-tfn Jan 4, 2024
03d19a1
Fixed docs, small change to gravitY_model home page
isaac-tfn Jan 4, 2024
f6ba348
Typo in maxdepth
isaac-tfn Jan 4, 2024
0d9e3c1
Gravity_model docs page not working
isaac-tfn Jan 4, 2024
00a1357
Possibly was an indentation error
isaac-tfn Jan 4, 2024
1c18540
Linting fixes
isaac-tfn Jan 5, 2024
e458993
Delete sphinx.yml
isaac-tfn Jan 5, 2024
d53b875
Merge branch 'main' into multi_tld_mvp
isaac-tfn Jan 5, 2024
1f3afc5
Black
isaac-tfn Jan 5, 2024
32a7192
Merge branch 'multi_tld_mvp' of https://github.com/Transport-for-the-…
isaac-tfn Jan 5, 2024
1a7c477
Pylint
isaac-tfn Jan 5, 2024
669d037
Add typing to _attempt_id
isaac-tfn Jan 5, 2024
e771396
Removed typings
isaac-tfn Jan 5, 2024
2b64d7d
infill tested on code generated cost matrix
isaac-tfn Jan 8, 2024
9a9295e
Black
isaac-tfn Jan 8, 2024
3f8207d
Fixed tests
isaac-tfn Jan 8, 2024
19305ca
Black again
isaac-tfn Jan 8, 2024
1c8f965
isort + black
isaac-tfn Jan 9, 2024
4019264
Fixed docstring
isaac-tfn Jan 9, 2024
f391a56
Fixed refactor in tests
isaac-tfn Jan 9, 2024
8ea0c7f
Pylint import order fixed
isaac-tfn Jan 9, 2024
febd9bb
Trying to suppress has-type warnings in mypy
isaac-tfn Jan 9, 2024
267f60b
Ignore type for strange mypy errors
isaac-tfn Jan 9, 2024
08de987
black
isaac-tfn Jan 9, 2024
9943d94
Updated release.md
isaac-tfn Jan 9, 2024
234ce4a
Short docstring for TestUtils
isaac-tfn Jan 9, 2024
3f1ab26
Update src/caf/distribute/gravity_model/multi_area.py
isaac-tfn Jan 9, 2024
0eccf14
typing for plot method
isaac-tfn Jan 9, 2024
f64011d
Merge branch 'multi_tld_mvp' of https://github.com/Transport-for-the-…
isaac-tfn Jan 9, 2024
95cf4f8
Update docs/conf.py
isaac-tfn Jan 9, 2024
ec135a9
Update docs/conf.py
isaac-tfn Jan 9, 2024
cffcf81
Merge branch 'multi_tld_mvp' of https://github.com/Transport-for-the-…
isaac-tfn Jan 9, 2024
16b7162
Fixing review notes
isaac-tfn Jan 9, 2024
60d4eac
Create git blame for isort commits
isaac-tfn Jan 10, 2024
a3a4057
Forgot to black docs
isaac-tfn Jan 10, 2024
dc9dad7
Added a couple of docstrings
isaac-tfn Jan 17, 2024
e0a3338
Imperative mood
isaac-tfn Jan 17, 2024
e9672b2
PR review changes
isaac-tfn Jan 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#Commit for isort + black
1c8f965bed3dc226550db96eefc54866f521d116
6 changes: 4 additions & 2 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,7 @@ sphinx:
# Optionally declare the Python requirements required to build your docs
python:
install:
- requirements: docs/requirements.txt
- requirements: requirements.txt
- method: pip
path: .
- requirements: docs/requirements.txt

2 changes: 1 addition & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ there.
Below, a brief summary of patches made since the previous version can be found.

### Next Release Notes
- Updated the license
- Multi-TLD gravity model added
14 changes: 4 additions & 10 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,20 +9,15 @@
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- Project information -----------------------------------------------------
import sys
import os
from pathlib import Path
import caf.distribute

dir_path = Path(__file__).parents[1]
source = dir_path / "src" / "caf" / "distribute"

sys.path.insert(0, os.path.abspath(str(source)))

# -- Project information -----------------------------------------------------
project = "caf.distribute"
copyright = "2023, Transport for the North"
author = "Transport for the North"
Expand Down Expand Up @@ -80,5 +75,4 @@
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]

autodoc_mock_imports = ["caf"]
autodoc_typehints = "description"
4 changes: 3 additions & 1 deletion docs/gravity_model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ Gravity Model sub-package
-------------------------

.. toctree::
:maxdepth: 1

core
single_area
single_area
multi_area
14 changes: 11 additions & 3 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,20 @@ Indices and tables
* :ref:`modindex`
* :ref:`search`

Contents
--------
Modules
-------

.. toctree::
:maxdepth: 1

cost_functions
furness
gravity_model
utils

Sub-Packages
------------

.. toctree::
:maxdepth: 1

gravity_model
2 changes: 2 additions & 0 deletions docs/multi_area.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

.. automodapi:: caf.distribute.gravity_model.multi_area
4 changes: 2 additions & 2 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Required libraries for a doc build
sphinx-automodapi>=0.16.0
pydata-sphinx-theme>=0.14.1
caf.distribute>=0.1.0
graphviz>=0.20.1
graphviz>=0.20.1
versioningit>=2.2.0, < 3.0.0
2 changes: 2 additions & 0 deletions docs/utils.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

.. automodapi:: caf.distribute.utils
15 changes: 14 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ dependencies = [
"pandas>=1.4.0",
"tqdm>=4.50.2",
"caf.toolkit>=0.2.1",
"matplotlib>=3.8.2"
]

[project.optional-dependencies]
Expand Down Expand Up @@ -74,6 +75,18 @@ disable = [
"use-list-literal",
]

[tool.isort]
profile = "black"
src_paths = ["src", "tests"]

skip = ["__init__.py", "_version.py"]
skip_glob = [".*"]

# Section commenting
import_heading_stdlib = "Built-Ins"
import_heading_thirdparty = "Third Party"
import_heading_firstparty = "Local Imports"

# Pylint settings
[tool.pylint.basic]
good-names = ["df", "fn", "ca", "tp", "to", "x", "x1", "x2", "i", "f", "q", "mu"]
Expand Down Expand Up @@ -102,7 +115,7 @@ add-ignore = [
# Tests
[tool.pytest.ini_options]
pythonpath = ["src"]
addopts = '--cov=caf --cov-report=xml -n auto'
addopts = '--cov=caf --cov-report=xml -n auto' # comment out this line to debug a test
BenTaylor-TfN marked this conversation as resolved.
Show resolved Hide resolved

[tool.coverage.report]
include_namespace_packages = true
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ numpy>=1.19.0
pandas>=1.4.0
tqdm>=4.50.2
caf.toolkit>=0.2.1
matplotlib>=3.8.2
9 changes: 2 additions & 7 deletions src/caf/distribute/cost_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,14 @@

# Built-Ins
import enum
import random
import inspect
import logging

from typing import Any
from typing import Mapping
from typing import Optional
from typing import Callable
import random
from typing import Any, Callable, Mapping, Optional

# Third Party
import numpy as np

# Local Imports
# pylint: disable=import-error,wrong-import-position
from caf.toolkit import math_utils

Expand Down
4 changes: 2 additions & 2 deletions src/caf/distribute/furness.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
# Third Party
import numpy as np

# Local Imports
# pylint: disable=import-error,wrong-import-position

# pylint: enable=import-error,wrong-import-position
Expand Down Expand Up @@ -74,6 +73,7 @@ def doubly_constrained_furness(
The Root Mean Squared Error difference achieved before exiting
"""
# pylint: disable=too-many-locals
# TODO(MB) Incorporate Nhan's furnessing optimisations
# Error check
if seed_vals.shape != (len(row_targets), len(col_targets)):
raise ValueError(
Expand Down Expand Up @@ -134,7 +134,7 @@ def doubly_constrained_furness(
# Calculate the diff - leave early if met
row_diff = (row_targets - np.sum(furnessed_mat, axis=1)) ** 2
col_diff = (col_targets - np.sum(furnessed_mat, axis=0)) ** 2
cur_rmse = (np.sum(row_diff + col_diff) / n_vals) ** 0.5
cur_rmse = ((np.sum(row_diff) + np.sum(col_diff)) / n_vals) ** 0.5
BenTaylor-TfN marked this conversation as resolved.
Show resolved Hide resolved
if cur_rmse < tol:
early_exit = True
break
Expand Down
7 changes: 5 additions & 2 deletions src/caf/distribute/gravity_model/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@
from caf.distribute.gravity_model.core import GravityModelResults
from caf.distribute.gravity_model.core import GravityModelRunResults
from caf.distribute.gravity_model.core import GravityModelCalibrateResults
from caf.distribute.gravity_model.multi_area import (
MultiCostDistribution,
MultiDistInput,
)

# Models
from caf.distribute.gravity_model.single_area import SingleAreaGravityModelCalibrator

# import MultiTLDGravityModel
from caf.distribute.gravity_model.multi_area import MultiAreaGravityModelCalibrator
Loading
Loading