Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge develop-main to develop-cloud #562

Draft
wants to merge 1 commit into
base: develop-cloud
Choose a base branch
from
Draft

Merge develop-main to develop-cloud #562

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
project = "OptiNiSt"
copyright = f"{datetime.today().year}, OIST"
author = ""
release = "1.2.1"
release = "1.3.0"

# -- readthedocs -------------------------------------------------------------
on_rtd = os.environ.get("READTHEDOCS", None) == "True"
Expand Down
15 changes: 12 additions & 3 deletions docs/specifications/algorithm_nodes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ OptiNiSt includes a variety of third-party calcium (Ca<sup>2+</sup>) imaging sof
###### lccd_detect
- **Description:** Low Computational-cost Cell Detection
- **Input:** ImageData
- **Output:** FluoData (time series for each detected cell), RoiData (spatial masks for each detected cell)
- **Output:** FluoData, IsCellData
- **Parameters:**
- **blob_detector:** Initial step that identifies potential cells or "blobs" in each frame of the video.
- **filtersize1** [int (filtersize1 <= Time / 2), default: 100]: Size of the larger temporal filter for smoothing.
Expand Down Expand Up @@ -498,10 +498,19 @@ OptiNiSt includes a variety of third-party calcium (Ca<sup>2+</sup>) imaging sof
###### Fluo from HDF5
- **Description:** Extracts fluorescence data from HDF5 file and transposes for visualization
- **Input:** FluoData, HDF5Data
- If using optinist nwb format:
- processing/ophys/suite2p_roi_UNIQUE-ID/Fluorescence/data
- processing/ophys/caiman_cnmf_UNIQUE-ID/Fluorescence/data
- **Output:** FluoData

###### ROI from HDF5
- **Description:** Extracts ROI data from HDF5 file and prepares it for visualization
- **Input:** ImageData, HDF5Data, IscellData
- **Output:** RoiData

- If using optinist nwb format:
cell_img:
- processing/optinist/suite2p_roi_UNIQUE-ID_all_roi_img/data
- processing/optinist/caiman_cnmf_UNIQUE-ID_all_roi_img/data
iscell:
- processing/ophys/ImageSegmentation/suite2p_roi_UNIQUE-ID/iscell
- processing/ophys/ImageSegmentation/caiman_cnmf_UNIQUE-ID/iscell
- **Output:** IscellData
6 changes: 3 additions & 3 deletions frontend/build/asset-manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
{
"files": {
"main.css": "/static/css/main.edcdbd9f.css",
"main.js": "/static/js/main.2ba982aa.js",
"main.js": "/static/js/main.19be1f34.js",
"static/js/488.8edbc83a.chunk.js": "/static/js/488.8edbc83a.chunk.js",
"static/media/logo.png": "/static/media/logo.e44efe3246aecc6d6219.png",
"static/media/index.cjs": "/static/media/index.d64cca1358b36db061fb.cjs",
"index.html": "/index.html",
"main.edcdbd9f.css.map": "/static/css/main.edcdbd9f.css.map",
"main.2ba982aa.js.map": "/static/js/main.2ba982aa.js.map",
"main.19be1f34.js.map": "/static/js/main.19be1f34.js.map",
"488.8edbc83a.chunk.js.map": "/static/js/488.8edbc83a.chunk.js.map"
},
"entrypoints": [
"static/css/main.edcdbd9f.css",
"static/js/main.2ba982aa.js"
"static/js/main.19be1f34.js"
]
}
2 changes: 1 addition & 1 deletion frontend/build/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="/static/favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="Web site created using create-react-app"/><link rel="manifest" href="/static/manifest.json"/><title>OptiNiSt</title><script defer="defer" src="/static/js/main.2ba982aa.js"></script><link href="/static/css/main.edcdbd9f.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
<!doctype html><html lang="en"><head><meta charset="utf-8"/><link rel="icon" href="/static/favicon.ico"/><meta name="viewport" content="width=device-width,initial-scale=1"/><meta name="theme-color" content="#000000"/><meta name="description" content="Web site created using create-react-app"/><link rel="manifest" href="/static/manifest.json"/><title>OptiNiSt</title><script defer="defer" src="/static/js/main.19be1f34.js"></script><link href="/static/css/main.edcdbd9f.css" rel="stylesheet"></head><body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div></body></html>
221 changes: 221 additions & 0 deletions frontend/build/static/js/main.19be1f34.js
View file
Open in desktop

Large diffs are not rendered by default.

0 ...ld/static/js/main.2ba982aa.js.LICENSE.txt → ...ld/static/js/main.19be1f34.js.LICENSE.txt
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion ...tend/build/static/js/main.2ba982aa.js.map → ...tend/build/static/js/main.19be1f34.js.map
View file
Open in desktop

Large diffs are not rendered by default.

221 changes: 0 additions & 221 deletions frontend/build/static/js/main.2ba982aa.js
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion frontend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "optinist",
"version": "1.2.1",
"version": "1.3.0",
"engines": {
"node": ">=20.0.0"
},
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[tool.poetry]
name = "optinist"
description = "Calcium Imaging Pipeline Tool"
version = "1.2.1"
version = "1.3.0"
license = "GPL-3.0"
authors = ["OIST <[email protected]>"]
readme = "README.md"
Expand Down
4 changes: 2 additions & 2 deletions sample_data/output/tutorial1/experiment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,15 @@ function:
started_at: '2023-11-22 16:09:06'
success: success
unique_id: eta_jntqpbkvwv
input_0:
input_ab1mmvt2ky:
finished_at: '2023-11-22 16:07:03'
hasNWB: false
message: null
name: sample_mouse2p_image.tiff
outputPaths: null
started_at: '2023-11-22 16:07:03'
success: success
unique_id: input_0
unique_id: input_ab1mmvt2ky
input_zdax4o54o0:
finished_at: '2023-11-22 16:07:03'
hasNWB: false
Expand Down
10 changes: 5 additions & 5 deletions sample_data/output/tutorial1/snakemake.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ rules:
input_zdax4o54o0: behaviors_data
iscell: iscell
type: eta
input_0:
input_ab1mmvt2ky:
hdf5Path: null
input:
- 1/sample_mouse2p_image.tiff
Expand Down Expand Up @@ -54,10 +54,10 @@ rules:
emission_lambda: 500.0
name: OpticalChannel
session_description: optinist
output: 1/tutorial1/input_0/sample_mouse2p_image.pkl
output: 1/tutorial1/input_ab1mmvt2ky/sample_mouse2p_image.pkl
params: {}
path: null
return_arg: input_0
return_arg: input_ab1mmvt2ky
type: image
input_zdax4o54o0:
hdf5Path: null
Expand Down Expand Up @@ -100,7 +100,7 @@ rules:
suite2p_file_convert_axgpevzu10:
hdf5Path: null
input:
- 1/tutorial1/input_0/sample_mouse2p_image.pkl
- 1/tutorial1/input_ab1mmvt2ky/sample_mouse2p_image.pkl
nwbfile: null
output: 1/tutorial1/suite2p_file_convert_axgpevzu10/suite2p_file_convert.pkl
params:
Expand All @@ -111,7 +111,7 @@ rules:
nplanes: 1
path: suite2p/suite2p_file_convert
return_arg:
input_0: image
input_ab1mmvt2ky: image
type: suite2p_file_convert
suite2p_registration_ktrkq5sa71:
hdf5Path: null
Expand Down
12 changes: 6 additions & 6 deletions sample_data/output/tutorial1/workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
edgeDict:
? reactflow__edge-input_0input_0--image--ImageData-suite2p_file_convert_axgpevzu10suite2p_file_convert_axgpevzu10--image--ImageData
? reactflow__edge-input_ab1mmvt2kyinput_ab1mmvt2ky--image--ImageData-suite2p_file_convert_axgpevzu10suite2p_file_convert_axgpevzu10--image--ImageData
: animated: false
id: reactflow__edge-input_0input_0--image--ImageData-suite2p_file_convert_axgpevzu10suite2p_file_convert_axgpevzu10--image--ImageData
source: input_0
sourceHandle: input_0--image--ImageData
id: reactflow__edge-input_ab1mmvt2kyinput_ab1mmvt2ky--image--ImageData-suite2p_file_convert_axgpevzu10suite2p_file_convert_axgpevzu10--image--ImageData
source: input_ab1mmvt2ky
sourceHandle: input_ab1mmvt2ky--image--ImageData
style:
border: null
borderRadius: null
Expand Down Expand Up @@ -131,7 +131,7 @@ nodeDict:
padding: 0
width: 250
type: AlgorithmNode
input_0:
input_ab1mmvt2ky:
data:
fileType: image
hdf5Path: null
Expand All @@ -140,7 +140,7 @@ nodeDict:
path:
- sample_mouse2p_image.tiff
type: input
id: input_0
id: input_ab1mmvt2ky
position:
x: 50
y: 150
Expand Down
4 changes: 2 additions & 2 deletions sample_data/output/tutorial2/experiment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ function:
max_index: null
started_at: '2024-04-11 16:47:49'
finished_at: '2024-04-11 16:48:34'
input_0:
unique_id: input_0
input_cdgkgmc5oq:
unique_id: input_cdgkgmc5oq
name: sample_mouse2p_image.tiff
success: success
hasNWB: false
Expand Down
10 changes: 5 additions & 5 deletions sample_data/output/tutorial2/snakemake.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
rules:
caiman_mc_p4f6nsi9bh:
input:
- 1/tutorial2/input_0/sample_mouse2p_image.pkl
- 1/tutorial2/input_cdgkgmc5oq/sample_mouse2p_image.pkl
return_arg:
input_0: image
input_cdgkgmc5oq: image
params:
border_nan: copy
gSig_filt: null
Expand Down Expand Up @@ -36,12 +36,12 @@ rules:
hdf5Path: null
matPath: null
path: caiman/caiman_mc
input_0:
input_cdgkgmc5oq:
input:
- 1/sample_mouse2p_image.tiff
return_arg: input_0
return_arg: input_cdgkgmc5oq
params: {}
output: 1/tutorial2/input_0/sample_mouse2p_image.pkl
output: 1/tutorial2/input_cdgkgmc5oq/sample_mouse2p_image.pkl
type: image
nwbfile:
session_description: optinist
Expand Down
12 changes: 6 additions & 6 deletions sample_data/output/tutorial2/workflow.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,8 @@ nodeDict:
padding: 0
width: 250
borderRadius: 0
input_0:
id: input_0
input_cdgkgmc5oq:
id: input_cdgkgmc5oq
type: ImageFileNode
data:
label: sample_mouse2p_image.tiff
Expand Down Expand Up @@ -542,12 +542,12 @@ nodeDict:
width: 250
borderRadius: 0
edgeDict:
reactflow__edge-input_0input_0--image--ImageData-caiman_mc_p4f6nsi9bhcaiman_mc_p4f6nsi9bh--image--ImageData:
id: reactflow__edge-input_0input_0--image--ImageData-caiman_mc_p4f6nsi9bhcaiman_mc_p4f6nsi9bh--image--ImageData
reactflow__edge-input_cdgkgmc5oq--image--ImageData-caiman_mc_p4f6nsi9bhcaiman_mc_p4f6nsi9bh--image--ImageData:
id: reactflow__edge-input_cdgkgmc5oqinput_cdgkgmc5oq--image--ImageData-caiman_mc_p4f6nsi9bhcaiman_mc_p4f6nsi9bh--image--ImageData
type: buttonedge
animated: false
source: input_0
sourceHandle: input_0--image--ImageData
source: input_cdgkgmc5oq
sourceHandle: input_cdgkgmc5oq--image--ImageData
target: caiman_mc_p4f6nsi9bh
targetHandle: caiman_mc_p4f6nsi9bh--image--ImageData
style:
Expand Down
20 changes: 14 additions & 6 deletions studio/app/optinist/wrappers/lccd/lccd_detection.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

def lccd_detect(
mc_images: ImageData, output_dir: str, params: dict = None, **kwargs
) -> dict(fluorescence=FluoData, cell_roi=RoiData):
) -> dict(fluorescence=FluoData, iscell=IscellData):
from studio.app.optinist.wrappers.lccd.lccd_python.lccd import LCCD

function_id = ExptOutputPathIds(output_dir).function_id
Expand All @@ -38,9 +38,13 @@ def lccd_detect(
timeseries[i, :] = np.mean(reshapedD[roi[:, i] > 0, :], axis=0)

im = np.stack(roi_list)
im = im.astype(np.float64)
im[im == 0] = np.nan
im -= 1

empty_roi = np.full_like(im[0], np.nan)
roi_image = np.nanmax(im[iscell != 0], axis=0).astype(float)

timeseries_dff = np.ones([num_cell, num_frames]) * np.nan
for i in range(num_cell):
for k in range(num_frames):
Expand Down Expand Up @@ -71,21 +75,25 @@ def lccd_detect(
"table_name": "Fluorescence",
"region": list(range(len(timeseries))),
"name": "Fluorescence",
"data": timeseries.T,
"data": timeseries,
"unit": "lumens",
}
}
}

info = {
"dff": FluoData(timeseries_dff, file_name="dff"),
"fluorescence": FluoData(timeseries, file_name="fluorescence"),
"iscell": IscellData(iscell),
"all_roi": RoiData(roi_image, output_dir=output_dir, file_name="all_roi"),
"cell_roi": RoiData(
np.nanmax(im[iscell != 0], axis=0),
roi_image,
output_dir=output_dir,
file_name="cell_roi",
),
"fluorescence": FluoData(timeseries, file_name="fluorescence"),
"dff": FluoData(timeseries_dff, file_name="dff"),
"iscell": IscellData(iscell),
"non_cell_roi": RoiData(
empty_roi, output_dir=output_dir, file_name="non_cell_roi"
),
"edit_roi_data": EditRoiData(images=mc_images.data, im=im),
"nwbfile": nwbfile,
}
Expand Down
9 changes: 5 additions & 4 deletions studio/app/optinist/wrappers/optinist/visualize_utils/roi_from_hdf5.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,23 @@ def roi_from_hdf5(
iscell: IscellData = None,
params: dict = None,
**kwargs
) -> dict():
) -> dict(iscell=IscellData):
if iscell is not None:
iscell = iscell.data
iscell_data = iscell.data
return {
"iscell": IscellData(iscell_data, file_name="iscell"),
"all_roi": RoiData(
np.nanmax(cell_img.data, axis=0),
output_dir=output_dir,
file_name="roi",
),
"non_cell_roi": RoiData(
np.nanmax(cell_img.data[iscell == 0], axis=0),
np.nanmax(cell_img.data[iscell_data == 0], axis=0),
output_dir=output_dir,
file_name="noncell_roi",
),
"cell_roi": RoiData(
np.nanmax(cell_img.data[iscell != 0], axis=0),
np.nanmax(cell_img.data[iscell_data != 0], axis=0),
output_dir=output_dir,
file_name="cell_roi",
),
Expand Down
Loading