Neptune Ai - Systematica - Blockforce Capital

`NeptuneAnalyzer`

NeptuneAnalyzer(
    project: str = 'systematica',
    api_token: str = None,
)

A class for analyzing Neptune runs and fetching experiment data. Method generated by attrs for class NeptuneAnalyzer.

Static methods

`combine_trials`

combine_trials(
    trials: Dict[str, Any],
    metrics: List[str] = None,
) ‑> pandas.core.frame.DataFrame

Combine trials into a pd.DataFrame object. Parameters:

Name	Type	Default	Description
`trials`	`tp.Kwargs`	`--`	Dictionary containing trial data.
`metric`	`tp.List[str]`	`None`	List of metrics to fall back to if values is not in the trials. If `None`, a single ‘metrics’ column with NaN is added when ‘values’ is missing. Defaults to `None`.

Returns:

Type	Description
`pd.DataFrame`	DataFrame containing combined trial data.

Instance variables

api_token: str: Neptune API token, e.g. “your_api_token”.
project: str: Neptune project name, e.g. “user_name/project_name”.

Methods

`run_context`

run_context(
    self,
    **kwargs,
)

Context manager to initialize and stop a Neptune run. Parameters:

Name	Type	Default	Description
`kwargs`	`tp.Kwargs`	`--`	Additional arguments to pass to `neptune.init_run` (e.g., `with_id`, `mode`).

Yields:

Type	Description
`neptune.Run`	The initialized Neptune run object.

`clear_cache`

clear_cache(
    self,
)

Clear the cache for _fetch_neptune_metadata.

`best_metrics`

best_metrics(
    self,
    run_ids: str | List[str],
) ‑> pandas.core.frame.DataFrame

Fetch best metrics from Neptune runs. Parameters:

Name	Type	Default	Description
`run_ids`	`str`	`--`	The ID(s) of the Neptune run(s) to fetch.

Returns:

Type	Description
`pd.DataFrame`	DataFrame with metrics as index and run_ids as columns.

`best_params`

best_params(
    self,
    run_ids: str | List[str],
) ‑> pandas.core.frame.DataFrame

Fetch best parameters from Neptune runs. Parameters:

Name	Type	Default	Description
`run_ids`	`str`	`--`	The ID(s) of the Neptune run(s) to fetch.

Returns:

Type	Description
`pd.DataFrame`	DataFrame with parameters as index and run_ids as columns.

`trial_params`

trial_params(
    self,
    run_id: str,
) ‑> pandas.core.frame.DataFrame

Fetch trial parameters from Neptune runs. Metadata is queried with the below:

metadata_tag=trials/trials/0/distributions (Primary)
metadata_tag=trials/trials (Fallback)

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`pd.DataFrame`	`pd.DataFrame` with parameters as index and `run_ids` as columns.

`get_best_trials`

get_best_trials(
    self,
    run_ids: str | List[str],
) ‑> pandas.core.frame.DataFrame

Fetch the best trials from Neptune runs. Parameters:

Name	Type	Default	Description
`run_ids`	`str`	`--`	The ID(s) of the Neptune run(s) to fetch.

Returns:

Type	Description
`pd.DataFrame`	Trial data, including ‘run_id’ and ‘number’.

`get_all_trials`

get_all_trials(
    self,
    run_ids: str | List[str],
    is_file: bool = True,
) ‑> pandas.core.frame.DataFrame

Fetch all trials from Neptune runs. Parameters:

Name	Type	Default	Description
`run_ids`	`str`	`--`	The ID(s) of the Neptune run(s) to fetch.
`is_file`	`bool`	`True`	Flag to fetch compressed file trials through ‘trials/compressed_trials’ if True. Fetch ‘trials/trials’ otherwise. Defaults to True.

Returns:

Type	Description
`pd.DataFrame`	All trial data, including ‘run_id’ and ‘number’.

`metrics`

metrics(
    self,
    run_id: str,
) ‑> List[str]

List metric names from a Neptune run. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`list of str`	List of metric names.

`params`

params(
    self,
    run_id: str,
) ‑> List[str]

List parameter names from a Neptune run. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`list of str`	List of parameter names.

`all_params`

all_params(
    self,
    run_id: str,
) ‑> List[str]

List parameter names from a Neptune run. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`list of str`	List of parameter names.

`run_composer`

run_composer(
    self,
    run_id: str,
    trial_selector: str | int | systematica.portfolio.trial_selectors.base.BaseTrialSelector = None,
    loader: vectorbtpro.data.base.Data | Callable = None,
    loader_kwargs: Dict[str, Any] = None,
    column_stack: bool = False,
    group_by: bool = True,
) ‑> systematica.portfolio.analyzers.base.PortfolioAnalyzer

Run portfolio analytics from run_id. New generation logging is supported. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`trial_selector`	`str`	`None`	Custom parameter selection. If None, retrieve `best/params` from neptune. if int, retrieve trial number. If `BaseTrialSelector`, retrieve params based on algorithm. Defaults to None.
`loader`	`vbt.Data`	`None`	Data loader instance. If None, defaults to `load_clean_data`. Defaults to None.
`loader_kwargs`	`tp.Kwargs`	`None`	Additional loader parameters. Note that timeframe, start and end are already fetched from the neptune pipeline field. Defaults to None.
`column_stack`	`bool`	`False`	Compute `vbt.PF.column_stack` to combine portfolio when several trial numbers are passed. Default to False.
`group_by`	`bool`	`True`	Group strategy when `column_stack` is True. If False, compute individual backtests. Combined all strategy into a single run otherwise. Defaults to True.

Returns:

Type	Description
`PortfolioAnalyzer`	Class with inherited `PortfolioAnalizer`.

`all_combination`

all_combination(
    self,
    run_id: str,
) ‑> List[str]

Generate combinations of metric and parameter names. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`list of tuple`	List of (metric, param) tuples.

`param_combination`

param_combination(
    self,
    run_id: str,
) ‑> List[str]

Generate permutations of parameter names. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`list of tuple`	List of (metric, param) tuples.

`optuna_study`

optuna_study(
    self,
    run_id: str,
) ‑> optuna.study.study.Study

Loads Optuna study from an existing Neptune run. Loading mechanics depend on the study storage type used during the run:

If the study used InMemoryStorage, it will be loaded from the logged pickled study object.
If the study used database storage, it will be loaded from the logged database URL.

To resume an existing run, you need the run ID. It is stored in the run’s sys/id field. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`optuna.Study`	The Optuna study object.

`optuna_trials_dataframe`

optuna_trials_dataframe(
    self,
    run_id: str,
) ‑> pandas.core.frame.DataFrame

Export optuna trials as a pandas DataFrame. The DataFrame provides various features to analyze studies. It is also useful to draw a histogram of objective values and to export trials as a CSV file. If there are no trials, an empty pd.DataFrame is returned. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`pd.DataFrame`	The Optuna study object.

`optuna_best_trials`

optuna_best_trials(
    self,
    run_id: str,
) ‑> pandas.core.frame.DataFrame

Export optuna best trials as a pandas DataFrame. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`pd.DataFrame`	The Optuna study object.

`stats`

stats(
    self,
    run_id: str,
) ‑> pandas.core.series.Series

Compute statistics from a Neptune run. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`pd.Series`	Placeholder return value.

`plot_pareto_front`

plot_pareto_front(
    self,
    run_id: str,
    metrics: List[str] = None,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot pareto front. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`metrics`	`tp.List[str]`	`None`	Metrics to display. If `None`, All metrics are displayed. Defaults to `None`.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_contour`

plot_contour(
    self,
    run_id: str,
    params: List[str],
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the parameter relationship as contour plot in a study. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`params`	`tp.List[str]`	`--`	Parameter list to visualize.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_param_importances`

plot_param_importances(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot parameter importance.

An importance evaluator object that specifies which algorithm to base the importance assessment on, defaults to FanovaImportanceEvaluator.

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_edf`

plot_edf(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the objective value EDF (empirical distribution function) of a study. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_optimization_history`

plot_optimization_history(
    self,
    run_id: str,
    error_bar: bool = False,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot optimization history of all trials in a study. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`error_bar`	`bool`	`False`	A flag to show the error bar.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_parallel_coordinate`

plot_parallel_coordinate(
    self,
    run_id: str,
    metric: str,
    params: List[str] = None,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the high-dimensional parameter relationships in a study. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`metric`	`str`	`--`	Target metric.
`params`	`tp.List[str]`	`None`	Parameter list to visualize. if None, the default is all parameters.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_rank`

plot_rank(
    self,
    run_id: str,
    params: List[str] = None,
    n_columns: int = 4,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot parameter relations as scatter plots with colors indicating ranks of target value.

Trials missing the specified parameters will not be plotted.

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`params`	`tp.List[str]`	`None`	Parameter list to visualize. if None, the default is all parameters.
`n_columns`	`int`	`4`	When n_params is more than 3, the number of columns in the grid. The total number of subplots will be split into `n_columns` columns. layout_kwargs, tp.Kwargs Additional layout parameters.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_slice`

plot_slice(
    self,
    run_id: str,
    params: List[str] = None,
    n_columns: int = 4,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the parameter relationship as a slice plot in a study. Trials missing the specified parameters will not be plotted.

The slice plot is useful for visualizing the relationship between hyperparameters and the objective function values.It can help you:

Identify Parameter Sensitivity: Shows which hyperparameters have a strong influence on performance.
Detect Bad Regions: Highlights ranges where performance is consistently poor.
Spot Non-linear Patterns: You might notice trends.
Debug Optimization: If you see no pattern or strange clumping, it might indicate:
- A bad search space.
- Problems with the objective function.
Guide Future Searches: Helps you refine the search space for better performance in future runs.

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`params`	`tp.List[str]`	`None`	Parameter list to visualize. If None, the default is all parameters.
`n_columns`	`int`	`4`	When n_params is more than 3, the number of columns in the grid. The total number of subplots will be split into `n_columns` columns.
`layout_kwargs`	`tp.Kwargs`	`--`	Additional layout parameters.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_hypervolume_history`

plot_hypervolume_history(
    self,
    run_id: str,
    reference_point: Sequence[float] = None,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot hypervolume history of all trials in a study. This function is only applicable for multi-objective optimization studies. It computes the hypervolume of the Pareto front at each trial and plots the hypervolume history. The hypervolume is a measure of the volume of the dominated region in the objective space, defined by the reference point. The higher the hypervolume, the better the Pareto front.

Study must be multi-objective. For single-objective optimization, please use plot_optimization_history instead.

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`reference_point`	`tp.Sequence[float]`	`None`	A reference point to use for hypervolume computation. The dimension of the reference point must be the same as the number of objectives. If None, takes the min of each metrics to ensure the reference point is lower than all Pareto solutions and defines the dominated region and create measurable volume. Important note, the default value implied all objectives are minimized. Optuna transforms the objective values and the reference point using `study.directions`. Defaults to `None`.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_timeline`

plot_timeline(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the timeline of a study. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_intermediate_values`

plot_intermediate_values(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot intermediate values of all trials in a study. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_terminator_improvement`

plot_terminator_improvement(
    self,
    run_id: str,
    plot_error: bool = False,
    min_n_trials: int = 20,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the potentials for future objective improvement. This function does not support multi-objective optimization study.

This function visualizes the objective improvement potentials, evaluated with improvement_evaluator. It helps to determine whether we should continue the optimization or not. You can also plot the error evaluated with error_evaluator if the plot_error argument is set to True. Note that this function may take some time to compute the improvement potentials. The improvement_evaluator defaults to RegretBoundEvaluator and the error_evaluator to CrossValidationErrorEvaluator.

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`plot_error`	`bool`	`False`	A flag to show the error. If it is set to `True`, errors evaluated by error_evaluator are also plotted as line graph. Defaults to `False`.
`min_n_trials`	`int`	`20`	The minimum number of trials before termination is considered. Terminator improvements for trials below this value are shown in a lighter color. Defaults to `20`.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_params_correlation`

plot_params_correlation(
    self,
    run_id,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot parameters correlation matrix. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_metrics_correlation`

plot_metrics_correlation(
    self,
    run_id,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot metrics correlation matrix. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_metrics`

plot_metrics(
    self,
    run_id: str,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a histogram matrix of metrics for a Neptune run. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`layout_kwargs`	`tp.Kwargs`	`--`	Additional layout arguments for the plot.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_params`

plot_params(
    self,
    run_id: str,
    n_columns: int = 3,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a scatter matrix of parameters for a Neptune run. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`n_columns`	`int`	`3`	Number of columns in the scatter matrix. Defaults to 3.
`layout_kwargs`	`tp.Kwargs`	`--`	Additional layout arguments for the plot.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_density_heatmap`

plot_density_heatmap(
    self,
    run_id: str,
    metric: str,
    param: str,
    **kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a density heatmap for a metric and parameter. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`metric`	`str`	`--`	The metric to plot.
`param`	`str`	`--`	The parameter to plot.
`kwargs`	`tp.Kwargs`	`--`	Additional arguments for the plot.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_density_contour`

plot_density_contour(
    self,
    run_id: str,
    metric: str,
    param: str,
    colorscale: List[str | Tuple[float]] = None,
    **kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a density contour for a metric and parameter. Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`metric`	`str`	`--`	The metric to plot.
`param`	`str`	`--`	The parameter to plot.
`colorscale`	`list`	`--`	The colorscale for the plot.
`kwargs`	`tp.Kwargs`	`--`	Additional arguments for the plot.

Returns:

Type	Description
`vbt.FigureWidget`	The generated plot.

`plot_frontier`

plot_frontier(
    self,
    run_id: str,
    x: str,
    y: str,
    list_trial_selectors: List[systematica.portfolio.trial_selectors.base.BaseTrialSelector] = None,
    remove_negative_values: bool = False,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot an efficient frontier.

This function plots the efficient frontier of a study, which is a graphical representation of the trade-off between risk and return. The efficient frontier is a curve that shows the optimal risk-return combinations for a given set of trials. The points on the curve represent the best possible trade-offs between risk and return, while points below the curve are suboptimal:

x is generally a risk metric (e.g., volatility), while y is a performance measure (e.g., returns).
The risk-adjusted optimized point is calculated as: risk_adj = (returns - risk_free_rate) / risks

Parameters:

Name	Type	Default	Description
`run_id`	`str`	`--`	The ID of the Neptune run to fetch.
`x`	`str`	`--`	X axis. Must be a metric from neptune run.
`y`	`str`	`--`	Y axis. Must be a metric from neptune run.
`list_trial_selectors`	`tp.List[BaseTrialSelector]`	`None`	Trial selector points added to the charts. Defaults to `None`.
`risk_free_rate`	`float`	`0.0`	Risk-free rate for volatility calculation. Defaults to 0.0.
`layout_kwargs`	`tp.Kwargs`	`--`	Additional layout parameters.

Returns:

Type	Description
`vbt.FigureWidget`	The generated efficient frontier plot with CML.

Overview

Analyzers

Composers

Frontier

Messengers

Metrics

Sandbox

Schedulers

Trackers

Transformers

Trial Selectors

Tuners

​NeptuneAnalyzer

​Static methods

​combine_trials

​Instance variables

​Methods

​run_context

​clear_cache

​best_metrics

​best_params

​trial_params

​get_best_trials

​get_all_trials

​metrics

​params

​all_params

​run_composer

​all_combination

​param_combination

​optuna_study

​optuna_trials_dataframe

​optuna_best_trials

​stats

​plot_pareto_front

​plot_contour

​plot_param_importances

​plot_edf

​plot_optimization_history

​plot_parallel_coordinate

​plot_rank

​plot_slice

​plot_hypervolume_history

​plot_timeline

​plot_intermediate_values

​plot_terminator_improvement

​plot_params_correlation

​plot_metrics_correlation

​plot_metrics

​plot_params

​plot_density_heatmap

​plot_density_contour

​plot_frontier

`NeptuneAnalyzer`

Static methods

`combine_trials`

Instance variables

Methods

`run_context`

`clear_cache`

`best_metrics`

`best_params`

`trial_params`

`get_best_trials`

`get_all_trials`

`metrics`

`params`

`all_params`

`run_composer`

`all_combination`

`param_combination`

`optuna_study`

`optuna_trials_dataframe`

`optuna_best_trials`

`stats`

`plot_pareto_front`

`plot_contour`

`plot_param_importances`

`plot_edf`

`plot_optimization_history`

`plot_parallel_coordinate`

`plot_rank`

`plot_slice`

`plot_hypervolume_history`

`plot_timeline`

`plot_intermediate_values`

`plot_terminator_improvement`

`plot_params_correlation`

`plot_metrics_correlation`

`plot_metrics`

`plot_params`

`plot_density_heatmap`

`plot_density_contour`

`plot_frontier`