Neptune Ai

systematica.portfolio.analyzers.neptune_ai

NeptuneAnalyzer

NeptuneAnalyzer(
    project: str = 'systematica',
    api_token: str = None,
)

A class for analyzing Neptune runs and fetching experiment data.

Method generated by attrs for class NeptuneAnalyzer.

Static methods

combine_trials

combine_trials(
    trials: Dict[str, Any],
    metrics: List[str] = None,
) ‑> pandas.core.frame.DataFrame

Combine trials into a pd.DataFrame object.

Parameters:

Name	Type	Default	Description
trials	tp.Kwargs	--	Dictionary containing trial data.
metric	tp.List[str]	None	List of metrics to fall back to if values is not in the trials. If None, a single ‘metrics’ column with NaN is added when ‘values’ is missing. Defaults to None.

Returns:

Type	Description
pd.DataFrame	DataFrame containing combined trial data.

Instance variables

• api_token: str: Neptune API token, e.g. “your_api_token”.

• project: str: Neptune project name, e.g. “user_name/project_name”.

Methods

run_context

run_context(
    self,
    **kwargs,
)

Context manager to initialize and stop a Neptune run.

Parameters:

Name	Type	Default	Description
kwargs	tp.Kwargs	--	Additional arguments to pass to neptune.init_run (e.g., with_id, mode).

Yields:

Type	Description
neptune.Run	The initialized Neptune run object.

clear_cache

clear_cache(
    self,
)

Clear the cache for _fetch_neptune_metadata.

best_metrics

best_metrics(
    self,
    run_ids: str | List[str],
) ‑> pandas.core.frame.DataFrame

Fetch best metrics from Neptune runs.

Parameters:

Name	Type	Default	Description
run_ids	str	--	The ID(s) of the Neptune run(s) to fetch.

Returns:

Type	Description
pd.DataFrame	DataFrame with metrics as index and run_ids as columns.

best_params

best_params(
    self,
    run_ids: str | List[str],
) ‑> pandas.core.frame.DataFrame

Fetch best parameters from Neptune runs.

Parameters:

Name	Type	Default	Description
run_ids	str	--	The ID(s) of the Neptune run(s) to fetch.

Returns:

Type	Description
pd.DataFrame	DataFrame with parameters as index and run_ids as columns.

trial_params

trial_params(
    self,
    run_id: str,
) ‑> pandas.core.frame.DataFrame

Fetch trial parameters from Neptune runs.

Metadata is queried with the below:

• metadata_tag=trials/trials/0/distributions (Primary)
• metadata_tag=trials/trials (Fallback)

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
pd.DataFrame	pd.DataFrame with parameters as index and run_ids as columns.

get_best_trials

get_best_trials(
    self,
    run_ids: str | List[str],
) ‑> pandas.core.frame.DataFrame

Fetch the best trials from Neptune runs.

Parameters:

Name	Type	Default	Description
run_ids	str	--	The ID(s) of the Neptune run(s) to fetch.

Returns:

Type	Description
pd.DataFrame	Trial data, including ‘run_id’ and ‘number’.

get_all_trials

get_all_trials(
    self,
    run_ids: str | List[str],
    is_file: bool = True,
) ‑> pandas.core.frame.DataFrame

Fetch all trials from Neptune runs.

Parameters:

Name	Type	Default	Description
run_ids	str	--	The ID(s) of the Neptune run(s) to fetch.
is_file	bool	True	Flag to fetch compressed file trials through ‘trials/compressed_trials’ if True. Fetch ‘trials/trials’ otherwise. Defaults to True.

Returns:

Type	Description
pd.DataFrame	All trial data, including ‘run_id’ and ‘number’.

metrics

metrics(
    self,
    run_id: str,
) ‑> List[str]

List metric names from a Neptune run.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
list of str	List of metric names.

params

params(
    self,
    run_id: str,
) ‑> List[str]

List parameter names from a Neptune run.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
list of str	List of parameter names.

all_params

all_params(
    self,
    run_id: str,
) ‑> List[str]

List parameter names from a Neptune run.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
list of str	List of parameter names.

run_composer

run_composer(
    self,
    run_id: str,
    trial_selector: str | int | systematica.portfolio.trial_selectors.base.BaseTrialSelector = None,
    loader: vectorbtpro.data.base.Data | Callable = None,
    loader_kwargs: Dict[str, Any] = None,
    column_stack: bool = False,
    group_by: bool = True,
) ‑> systematica.portfolio.analyzers.base.PortfolioAnalyzer

Run portfolio analytics from run_id.

New generation logging is supported.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
trial_selector	str	None	Custom parameter selection. If None, retrieve best/params from neptune. if int, retrieve trial number. If BaseTrialSelector, retrieve params based on algorithm. Defaults to None.
loader	vbt.Data	None	Data loader instance. If None, defaults to load_clean_data. Defaults to None.
loader_kwargs	tp.Kwargs	None	Additional loader parameters. Note that timeframe, start and end are already fetched from the neptune pipeline field. Defaults to None.
column_stack	bool	False	Compute vbt.PF.column_stack to combine portfolio when several trial numbers are passed. Default to False.
group_by	bool	True	Group strategy when column_stack is True. If False, compute individual backtests. Combined all strategy into a single run otherwise. Defaults to True.

Returns:

Type	Description
PortfolioAnalyzer	Class with inherited PortfolioAnalizer.

all_combination

all_combination(
    self,
    run_id: str,
) ‑> List[str]

Generate combinations of metric and parameter names.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
list of tuple	List of (metric, param) tuples.

param_combination

param_combination(
    self,
    run_id: str,
) ‑> List[str]

Generate permutations of parameter names.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
list of tuple	List of (metric, param) tuples.

optuna_study

optuna_study(
    self,
    run_id: str,
) ‑> optuna.study.study.Study

Loads Optuna study from an existing Neptune run.

Loading mechanics depend on the study storage type used during the run:

• If the study used InMemoryStorage, it will be loaded from the logged pickled study object.
• If the study used database storage, it will be loaded from the logged database URL.

To resume an existing run, you need the run ID. It is stored in the run’s sys/id field.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
optuna.Study	The Optuna study object.

optuna_trials_dataframe

optuna_trials_dataframe(
    self,
    run_id: str,
) ‑> pandas.core.frame.DataFrame

Export optuna trials as a pandas DataFrame.

The DataFrame provides various features to analyze studies. It is also useful to draw a histogram of objective values and to export trials as a CSV file. If there are no trials, an empty pd.DataFrame is returned.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
pd.DataFrame	The Optuna study object.

optuna_best_trials

optuna_best_trials(
    self,
    run_id: str,
) ‑> pandas.core.frame.DataFrame

Export optuna best trials as a pandas DataFrame.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
pd.DataFrame	The Optuna study object.

stats

stats(
    self,
    run_id: str,
) ‑> pandas.core.series.Series

Compute statistics from a Neptune run.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
pd.Series	Placeholder return value.

plot_pareto_front

plot_pareto_front(
    self,
    run_id: str,
    metrics: List[str] = None,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot pareto front.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
metrics	tp.List[str]	None	Metrics to display. If None, All metrics are displayed. Defaults to None.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_contour

plot_contour(
    self,
    run_id: str,
    params: List[str],
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the parameter relationship as contour plot in a study.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
params	tp.List[str]	--	Parameter list to visualize.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_param_importances

plot_param_importances(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot parameter importance.

An importance evaluator object that specifies which algorithm to base the importance assessment on, defaults to FanovaImportanceEvaluator.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_edf

plot_edf(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the objective value EDF (empirical distribution function) of a study.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_optimization_history

plot_optimization_history(
    self,
    run_id: str,
    error_bar: bool = False,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot optimization history of all trials in a study.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
error_bar	bool	False	A flag to show the error bar.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_parallel_coordinate

plot_parallel_coordinate(
    self,
    run_id: str,
    metric: str,
    params: List[str] = None,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the high-dimensional parameter relationships in a study.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
metric	str	--	Target metric.
params	tp.List[str]	None	Parameter list to visualize. if None, the default is all parameters.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_rank

plot_rank(
    self,
    run_id: str,
    params: List[str] = None,
    n_columns: int = 4,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot parameter relations as scatter plots with colors indicating ranks of target value.

Trials missing the specified parameters will not be plotted.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
params	tp.List[str]	None	Parameter list to visualize. if None, the default is all parameters.
n_columns	int	4	When n_params is more than 3, the number of columns in the grid. The total number of subplots will be split into n_columns columns. layout_kwargs, tp.Kwargs Additional layout parameters.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_slice

plot_slice(
    self,
    run_id: str,
    params: List[str] = None,
    n_columns: int = 4,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the parameter relationship as a slice plot in a study.

Trials missing the specified parameters will not be plotted.

The slice plot is useful for visualizing the relationship between hyperparameters and the objective function values.

It can help you:

• Identify Parameter Sensitivity: Shows which hyperparameters have a strong influence on performance.
• Detect Bad Regions: Highlights ranges where performance is consistently poor.
• Spot Non-linear Patterns: You might notice trends.
• Debug Optimization: If you see no pattern or strange clumping, it might indicate:
  • A bad search space.
  • Problems with the objective function.
• Guide Future Searches: Helps you refine the search space for better performance in future runs.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
params	tp.List[str]	None	Parameter list to visualize. If None, the default is all parameters.
n_columns	int	4	When n_params is more than 3, the number of columns in the grid. The total number of subplots will be split into n_columns columns.
layout_kwargs	tp.Kwargs	--	Additional layout parameters.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_hypervolume_history

plot_hypervolume_history(
    self,
    run_id: str,
    reference_point: Sequence[float] = None,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot hypervolume history of all trials in a study.

This function is only applicable for multi-objective optimization studies. It computes the hypervolume of the Pareto front at each trial and plots the hypervolume history. The hypervolume is a measure of the volume of the dominated region in the objective space, defined by the reference point. The higher the hypervolume, the better the Pareto front.

Study must be multi-objective. For single-objective optimization, please use plot_optimization_history instead.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
reference_point	tp.Sequence[float]	None	A reference point to use for hypervolume computation. The dimension of the reference point must be the same as the number of objectives. If None, takes the min of each metrics to ensure the reference point is lower than all Pareto solutions and defines the dominated region and create measurable volume. Important note, the default value implied all objectives are minimized. Optuna transforms the objective values and the reference point using study.directions. Defaults to None.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_timeline

plot_timeline(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the timeline of a study.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_intermediate_values

plot_intermediate_values(
    self,
    run_id: str,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot intermediate values of all trials in a study.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_terminator_improvement

plot_terminator_improvement(
    self,
    run_id: str,
    plot_error: bool = False,
    min_n_trials: int = 20,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot the potentials for future objective improvement.

This function does not support multi-objective optimization study.

This function visualizes the objective improvement potentials, evaluated with improvement_evaluator. It helps to determine whether we should continue the optimization or not. You can also plot the error evaluated with error_evaluator if the plot_error argument is set to True. Note that this function may take some time to compute the improvement potentials. The improvement_evaluator defaults to RegretBoundEvaluator and the error_evaluator to CrossValidationErrorEvaluator.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
plot_error	bool	False	A flag to show the error. If it is set to True, errors evaluated by error_evaluator are also plotted as line graph. Defaults to False.
min_n_trials	int	20	The minimum number of trials before termination is considered. Terminator improvements for trials below this value are shown in a lighter color. Defaults to 20.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_params_correlation

plot_params_correlation(
    self,
    run_id,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot parameters correlation matrix.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_metrics_correlation

plot_metrics_correlation(
    self,
    run_id,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot metrics correlation matrix.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_metrics

plot_metrics(
    self,
    run_id: str,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a histogram matrix of metrics for a Neptune run.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
layout_kwargs	tp.Kwargs	--	Additional layout arguments for the plot.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_params

plot_params(
    self,
    run_id: str,
    n_columns: int = 3,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a scatter matrix of parameters for a Neptune run.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
n_columns	int	3	Number of columns in the scatter matrix. Defaults to 3.
layout_kwargs	tp.Kwargs	--	Additional layout arguments for the plot.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_density_heatmap

plot_density_heatmap(
    self,
    run_id: str,
    metric: str,
    param: str,
    **kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a density heatmap for a metric and parameter.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
metric	str	--	The metric to plot.
param	str	--	The parameter to plot.
kwargs	tp.Kwargs	--	Additional arguments for the plot.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_density_contour

plot_density_contour(
    self,
    run_id: str,
    metric: str,
    param: str,
    colorscale: List[str | Tuple[float]] = None,
    **kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot a density contour for a metric and parameter.

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
metric	str	--	The metric to plot.
param	str	--	The parameter to plot.
colorscale	list	--	The colorscale for the plot.
kwargs	tp.Kwargs	--	Additional arguments for the plot.

Returns:

Type	Description
vbt.FigureWidget	The generated plot.

plot_frontier

plot_frontier(
    self,
    run_id: str,
    x: str,
    y: str,
    list_trial_selectors: List[systematica.portfolio.trial_selectors.base.BaseTrialSelector] = None,
    remove_negative_values: bool = False,
    **layout_kwargs,
) ‑> vectorbtpro.utils.figure.FigureWidget

Plot an efficient frontier.

This function plots the efficient frontier of a study, which is a graphical representation of the trade-off between risk and return. The efficient frontier is a curve that shows the optimal risk-return combinations for a given set of trials. The points on the curve represent the best possible trade-offs between risk and return, while points below the curve are suboptimal:

• x is generally a risk metric (e.g., volatility), while y is a performance measure (e.g., returns).
• The risk-adjusted optimized point is calculated as: risk_adj = (returns - risk_free_rate) / risks

Parameters:

Name	Type	Default	Description
run_id	str	--	The ID of the Neptune run to fetch.
x	str	--	X axis. Must be a metric from neptune run.
y	str	--	Y axis. Must be a metric from neptune run.
list_trial_selectors	tp.List[BaseTrialSelector]	None	Trial selector points added to the charts. Defaults to None.
risk_free_rate	float	0.0	Risk-free rate for volatility calculation. Defaults to 0.0.
layout_kwargs	tp.Kwargs	--	Additional layout parameters.

Returns:

Type	Description
vbt.FigureWidget	The generated efficient frontier plot with CML.