Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Summer placement 2024 #14

Draft
wants to merge 113 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
d57326d
add thermal_working_demo
arjavp-ukaea Jun 22, 2024
1f9ae53
resolve merge conflicts with .gitignore
arjavp-ukaea Jun 22, 2024
da73a50
modify .npz and utils to generate/store data from 1 ground truth sim
arjavp-ukaea Jun 25, 2024
5e694a7
add .exodus files that are used to build dataset
arjavp-ukaea Jun 25, 2024
e597d28
add ground truth exodus file to be perturbed by mooseherder
arjavp-ukaea Jun 25, 2024
bd90218
update dataset class to only store temperature fields and y_data
arjavp-ukaea Jun 25, 2024
4a0feec
update to only include pycache and .vscode
arjavp-ukaea Jun 25, 2024
7234a10
add mlp regression model
arjavp-ukaea Jun 25, 2024
0c8731e
modify to use specific_heat + surf_heat_flux as material props to per…
arjavp-ukaea Jun 25, 2024
a2e8b2c
add requirements.txt
arjavp-ukaea Jun 25, 2024
a32ae7f
update to include venv and pyvale/
arjavp-ukaea Jun 25, 2024
c71b2db
remove extraneous input files and sim directories
arjavp-ukaea Jun 25, 2024
ff0c00b
include .ipynb checkpoints & exodus_files
arjavp-ukaea Jul 9, 2024
a0bbd15
add classes and change param to thermal_cond
arjavp-ukaea Jul 9, 2024
2643c03
add class imbalance and data uniqueness checks
arjavp-ukaea Jul 9, 2024
6394122
add all exodus files
arjavp-ukaea Jul 9, 2024
cc02b9e
save model state_dict of best training run
arjavp-ukaea Jul 9, 2024
7e075ce
fix bug causing mismatch between x and y data
arjavp-ukaea Jul 9, 2024
2a5ad51
update dataset to include 10,000 examples
arjavp-ukaea Jul 9, 2024
73657db
add .toml and init files for package setup
arjavp-ukaea Jul 22, 2024
ca41f2c
update to use sklearn's sobol with stratified sampling for equal clas…
arjavp-ukaea Jul 22, 2024
01de542
update to work with modified dataset_generator
arjavp-ukaea Jul 22, 2024
477cad1
update calculate_y_values to include tolerance factor for generating …
arjavp-ukaea Jul 22, 2024
6824197
update requirements.txt
arjavp-ukaea Jul 22, 2024
d800de0
update .gitignore
arjavp-ukaea Jul 22, 2024
cc831c3
remove exodus files from repo
arjavp-ukaea Jul 22, 2024
33c4792
remove exodus files from repo
arjavp-ukaea Jul 22, 2024
49e6828
add src, docs and data dirs and move existing files to relevant dirs
arjavp-ukaea Jul 22, 2024
d70ea37
move usage code to examples/
arjavp-ukaea Jul 22, 2024
0e41c89
create data loading example
arjavp-ukaea Jul 22, 2024
ce52bf2
create model training/eval example
arjavp-ukaea Jul 22, 2024
ccdfc5b
remove imports from template init.py
arjavp-ukaea Jul 22, 2024
3fce3bf
remove imports from template init.py
arjavp-ukaea Jul 22, 2024
a41ba69
update .gitignore to exclude build artifacts
arjavp-ukaea Jul 23, 2024
a1af257
add abstract base class for model init & training/eval pipelines
arjavp-ukaea Jul 23, 2024
9c2f554
update .toml to include additional dependencies
arjavp-ukaea Jul 23, 2024
2187e4b
update to include ml model classes
arjavp-ukaea Jul 23, 2024
7badd9d
update repr method to provide more metadata
arjavp-ukaea Jul 23, 2024
b092760
filter sobol warnings for reduced output to console
arjavp-ukaea Jul 23, 2024
e814c96
update running ml example to work with abstract base classes and logf…
arjavp-ukaea Jul 23, 2024
2d8e54e
update .gitignore to include logfile(s)
arjavp-ukaea Jul 23, 2024
1f01695
update .gitignore to include logfile(s)
arjavp-ukaea Jul 23, 2024
b40df4f
update README to include basic install info
arjavp-ukaea Jul 23, 2024
0e944b9
udpate README to include pytorch installation instructions
arjavp-ukaea Jul 23, 2024
f6eafc7
update README to include latest cuda instructions
arjavp-ukaea Jul 23, 2024
995e9fa
update package description in README
arjavp-ukaea Jul 23, 2024
c6a1a3b
update package description in README
arjavp-ukaea Jul 23, 2024
f9b6cb3
renamed class to SampleGenerator
arjavp-ukaea Jul 23, 2024
299a0e1
add inline comments to explain dataloading
arjavp-ukaea Jul 23, 2024
63a9bbb
update package name to icarus
arjavp-ukaea Jul 24, 2024
d514155
Merge pull request #8 from aurora-multiphysics/package-testing
arjavp-ukaea Jul 24, 2024
46f85b7
add LICENSE and README to root
arjavp-ukaea Jul 24, 2024
a5414af
correct classifier to GPLv3
arjavp-ukaea Jul 24, 2024
b372ca0
change package name to avoid pypi name conflict
arjavp-ukaea Jul 24, 2024
de1c5de
include package-testing env in .gitignore
arjavp-ukaea Jul 24, 2024
5f93b96
Merge pull request #9 from aurora-multiphysics/main
arjavp-ukaea Jul 24, 2024
8e9b43d
Revert "Merge pull request #8 from aurora-multiphysics/package-testing"
arjavp-ukaea Jul 24, 2024
55ff756
Merge pull request #10 from aurora-multiphysics/revert-9-main
arjavp-ukaea Jul 24, 2024
1045b25
add .testing-env/ to .gitignore
arjavp-ukaea Jul 24, 2024
10d3951
change package name to icarus
arjavp-ukaea Jul 24, 2024
d4723a2
move .npz file for easy loading from package
arjavp-ukaea Jul 24, 2024
e2842d8
update to include thermal_dataset_loading options
arjavp-ukaea Jul 24, 2024
7bd78cd
update .toml to add dependencies
arjavp-ukaea Jul 24, 2024
ed352e2
add datasets.py for convenient dataset loading
arjavp-ukaea Jul 24, 2024
3db981b
add moose-config tutorial for new users
arjavp-ukaea Jul 24, 2024
23c84b3
update example
arjavp-ukaea Jul 24, 2024
d607c9c
update example
arjavp-ukaea Jul 24, 2024
27b8b54
remove ground truth moose and exodus file
arjavp-ukaea Jul 24, 2024
281a638
change version number
arjavp-ukaea Jul 24, 2024
a82f67f
updated to reflect change in .npz file loc
arjavp-ukaea Jul 24, 2024
264a72c
updated to handle processing exisitng dataste or user_created one
arjavp-ukaea Jul 24, 2024
6f5043a
fix bug in _debug_output method
arjavp-ukaea Jul 24, 2024
380c757
update version to 0.1.8
arjavp-ukaea Jul 24, 2024
baf6315
fix another bug in logging softmax outputs for debugging
arjavp-ukaea Jul 30, 2024
c641bc2
add existing monoblock_dataset for dataloader
arjavp-ukaea Jul 30, 2024
eb7e74c
add example showing end-to- end use of icarus
arjavp-ukaea Jul 30, 2024
b1fdb02
add example for monoblock data processing
arjavp-ukaea Jul 30, 2024
c69d9b7
add skeleton example files for running mooseherder
arjavp-ukaea Jul 30, 2024
a7b9505
add example for generating monoblock sims
arjavp-ukaea Jul 30, 2024
52ebf59
add example generating thermal_plate sims
arjavp-ukaea Jul 30, 2024
7189212
add heading that describes example
arjavp-ukaea Jul 30, 2024
a14d268
add example gneerating thermal_plate_sims
arjavp-ukaea Jul 30, 2024
ef3e91e
add example building thermal_plate dataset
arjavp-ukaea Jul 30, 2024
04ada6a
add example building monoblock dataset
arjavp-ukaea Jul 30, 2024
23f24ec
update example on running mlp model
arjavp-ukaea Jul 30, 2024
1ee1e12
update .gitignore to track examples dir
arjavp-ukaea Jul 30, 2024
b9fc3a1
update icarus version
arjavp-ukaea Jul 30, 2024
1a82f28
add loading of monoblock dataset
arjavp-ukaea Jul 30, 2024
e94eadc
make utils.py class based
arjavp-ukaea Jul 30, 2024
e72b0f9
allow user to define ground_truth params
arjavp-ukaea Jul 30, 2024
c5cf695
add/update moose config setup script
arjavp-ukaea Jul 30, 2024
1a653fd
mechanical commit
barisc-ukaea Aug 16, 2024
f381113
Add files via upload
barisc-ukaea Aug 16, 2024
177e8b9
Delete src/icarus/mechanical/mechanical_cnn_model (2).py
barisc-ukaea Aug 16, 2024
510e902
Delete src/icarus/mechanical directory
barisc-ukaea Aug 16, 2024
be2f21b
Create test
barisc-ukaea Aug 16, 2024
175afd8
Create test
barisc-ukaea Aug 16, 2024
c3e1cfe
Add files via upload
barisc-ukaea Aug 16, 2024
8bded08
Merge pull request #12 from aurora-multiphysics/icarus-mechanical
barisc-ukaea Aug 16, 2024
3c61cb5
Revert "Icarus mechanical"
lukethehuman Aug 16, 2024
f1da691
Merge pull request #13 from aurora-multiphysics/revert-12-icarus-mech…
lukethehuman Aug 16, 2024
da4d104
Delete mechanical directory
barisc-ukaea Aug 16, 2024
1733194
update readme to include moose config file setup instructions
arjavp-ukaea Aug 20, 2024
d66b1cf
remove verbose README description and update package name to icarus-f…
arjavp-ukaea Aug 20, 2024
6750365
move examples directory to src/icarus
arjavp-ukaea Aug 23, 2024
83c1b88
add classes to init.py
arjavp-ukaea Aug 23, 2024
7d2de76
combine utils.py and sample_generator functionality
arjavp-ukaea Aug 23, 2024
500902b
remove extraneous normalisation functions
arjavp-ukaea Aug 23, 2024
e1c87d7
add nn dataclass for parameter passing
arjavp-ukaea Aug 23, 2024
ceed45e
add numpy docstring standard for funcs and classes
arjavp-ukaea Aug 23, 2024
9cfdd00
remove system specific paths to moose and proteus
arjavp-ukaea Aug 23, 2024
333b208
Merge pull request #11 from aurora-multiphysics/package-testing
arjavp-ukaea Aug 23, 2024
88cd106
Merge branch 'dev' into summer-placement
lukethehuman Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 14 additions & 158 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,160 +1,16 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
.vscode/
__pycache__/
.pyvale
pyvale
examples/2d_thermal_dataset/sim-workdir-1/*
examples/monoblock_dataset/sim-workdir-1/*
.ipynb_checkpoints/
.test-env
notebook_dataset_generator.ipynb
best_model.pth
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
dist/
model_training.log
.package-testing
.testing-env/
110 changes: 108 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,108 @@
# icarus
Machine learning for fusion simulation validation.
# Icarus
Machine learning for simulation validation. Icarus provide a set of machine learning tools that engineers can use to assess the agreement between an experiment and simulation; that is, to validate the simulation with experimental data.


## Installation


### Standard Installation (PyPI)

You can install icarus from PyPi as follows:

```
pip install icarus-fusion
```

### Developer Installation

Clone `icarus` to your local system and `cd` to the root directory of `icarus`. Ensure that your virtual environment is activated and run from the `icarus` root directory:

```
pip install -e .
```


### PyTorch

Icarus requires the latest stable version of PyTorch. The installation process varies depending on your hardware and operating system. Please follow the appropriate instructions below:

#### CPU Installation:

If you do not have access to NVIDIA GPUs, install the CPU version of PyTorch. Use the following commands based on your operating system:

- Windows/macOS:

```
pip3 install torch
```

- Linux:

```
pip3 install torch --index-url https://download.pytorch.org/whl/cpu
```

#### GPU Installation (NVIDIA CUDA):

If you have access to NVIDIA GPUs and want to leverage CUDA for faster computation, use these commands (note: CUDA is not available on MacOS):

- Linux:

```
pip3 install torch
```

- Windows:
```
pip3 install torch --index-url https://download.pytorch.org/whl/cu121
```

*Note: The CUDA version (`cu121` in this example) may change. Always check the official [PyTorch](https://pytorch.org/get-started/locally/) website for the most up-to-date installation instructions and CUDA version compatibility.*


#### Verifying Installation:

After installation, you can verify that PyTorch is installed correctly by running:

```python
import torch

print(torch.__version__)
print(torch.cuda.is_available()) # returns True if CUDA available and properly installed
```

## Getting Started

The examples folder includes a sequence of examples using `icarus` : to generate the dataset and train an ml model from the suite available on the generated data.

#### Moose Config File

here are instructions for hoe to setup the moose config file.

```python
from pathlib import Path
from mooseherder import MooseConfig


def main() -> None:
"""main: create moose config json
"""
config = {'main_path': Path.home()/ 'moose',
'app_path': Path.home() / 'proteus',
'app_name': 'proteus-opt'}

moose_config = MooseConfig(config)

save_path = Path.cwd() / 'example-moose-config.json'
moose_config.save_config(save_path)


if __name__ == "__main__":
main()
```

## Contributors

- Arjav Poudel, UK Atomic Energy Authority, (arjavp-ukaea)
- Baris Cavusoglu, UK Atomic Energy Authority, (barisc-ukaea)
- Luke Humphrey, UK Atomic Energy Authority, (lukethehuman)
5 changes: 5 additions & 0 deletions moose-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"main_path": "/path/to/moose",
"app_path": "/path/to/proteus",
"app_name": "proteus-opt"
}
17 changes: 14 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "icarus"
version = "0.1.0"
name = "icarus-fusion"
version = "0.1.2"
description = "A machine learning tool for fusion simulation validation."
authors = [
{ name = "arjavp-ukaea", email = "[email protected]" },
Expand All @@ -21,11 +21,22 @@ classifiers = [
requires-python = ">=3.11"
dependencies = [
"numpy>=1.26.0",
"netCDF4>=1.6.5",
"scikit-learn>=1.5.0",
"scikit-optimize>=0.10.2",
"mooseherder>=0.1.0",
"matplotlib>=3.8.0",
"tqdm>=4.66.4",
"scipy==1.11.3",
]

[project.urls]
"Repository" = "https://github.com/aurora-multiphysics/icarus"
"Issue Tracker" = "https://github.com/aurora-multiphysics/icarus/issues"

[tool.setuptools.packages.find]
where = ["src"]
where = ["src"]

[tool.setuptools.package-data]
"icarus" = ["data/*.npz"]
where = ["src"]
9 changes: 9 additions & 0 deletions src/icarus/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# -*- coding: utf-8 -*-
"""
icarus
"""

from icarus.sample_generator import CreateThermalDataset, SimulationParameters
from icarus.dataloader import ThermalModelDataset
from icarus.model import IcarusModel, NNParameters
from icarus.datasets import load_2d_thermal_dataset, load_monoblock_thermal_dataset
19 changes: 19 additions & 0 deletions src/icarus/create_moose_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
from pathlib import Path
from mooseherder import MooseConfig


def main() -> None:
"""main: create moose config json
"""
config = {'main_path': Path.home()/ 'moose',
'app_path': Path.home() / 'proteus',
'app_name': 'proteus-opt'}

moose_config = MooseConfig(config)

save_path = Path.cwd() / 'moose-config.json'
moose_config.save_config(save_path)


if __name__ == "__main__":
main()
Binary file added src/icarus/data/thermal_model_data.npz
Binary file not shown.
Binary file added src/icarus/data/thermal_monoblock_data.npz
Binary file not shown.
Loading