feature_store package

Submodules

ads.feature_store.query package

class ads.feature_store.query.filter.Filter(feature, condition, value, **kwargs)

Bases: Builder

Represents a filter for a query.

CONST_CONDITION = 'condition'

CONST_FEATURE = 'feature'

CONST_VALUE = 'value'

property condition

property feature

to_dict()

Converts the object to dictionary with kind, type and spec as keys.

Parameters:

**kwargs (Dict) –

The additional arguments. - filter_by_attribute_map: bool

If True, then in the result will be included only the fields presented in the attribute_map.

property value

with_condition(condition)

Sets the condition to apply to the feature.

Parameters:

condition (str) – The condition to apply to the feature.

with_feature(feature)

Sets the feature to filter on.

Parameters:

feature (str) – The feature to filter on.

with_value(value)

Sets the value to filter for.

Parameters:

value (str) – The value to filter for.

class ads.feature_store.query.filter.Logic(type, left_f=None, right_f=None, left_l=None, right_l=None, **kwargs)

Bases: Builder

A class representing a logical operation on filters.

AND = 'AND'

classmethod And(left_f=None, right_f=None, left_l=None, right_l=None)

CONST_LEFT_FILTER = 'leftFilter'

CONST_LEFT_LOGIC = 'leftLogic'

CONST_RIGHT_FILTER = 'rightFilter'

CONST_RIGHT_LOGIC = 'rightLogic'

CONST_TYPE = 'type'

OR = 'OR'

classmethod Or(left_f=None, right_f=None, left_l=None, right_l=None)

SINGLE = 'SINGLE'

classmethod Single(left_f)

property left_filter

property left_logic

property right_filter

property right_logic

to_dict()

Converts the object to dictionary with kind, type and spec as keys.

Parameters:

**kwargs (Dict) –

The additional arguments. - filter_by_attribute_map: bool

If True, then in the result will be included only the fields presented in the attribute_map.

property type

The type of the object as showing in YAML.

This implementation returns the class name with the first letter coverted to lower case.

with_left_filter(left_filter)

Sets the left filter of the logic.

Parameters:

left_filter: Filter or None

The left filter of the logic.

with_left_logic(left_logic)

Sets the left logic of the operation.

Parameters:

left_logic: Logic or None

The left logic of the operation.

with_right_filter(right_filter)

Sets the right filter of the logic.

Parameters:

right_filter: Filter or None

The right filter of the logic.

with_right_logic(right_logic)

Sets the right logic of the operation.

Parameters:

right_logic: Logic or None

The right logic of the operation.

with_type(type)

Sets the type of the logic.

Parameters:

type: str

A string representing the type of the logic to be performed.

class ads.feature_store.query.join.Join(query, on, left_on, right_on, join_type: JoinType = JoinType.INNER)

Bases: Builder

The Join class is used to join two tables or datasets.

Parameters:

• query (The table or dataset to join.) –

• left_on (A list of column names from the left table to join on.) –

• right_on (A list of column names from the right table to join on.) –

• (JoinType) (join_type) –

with_query(query: 'Query') → "Join": Sets the query attribute.

with_left_on(left_on: List[str]) → "Join": Sets the left_on attribute.

with_right_on(right_on: List[str]) → "Join": Sets the right_on attribute.

with_join_type(join_type: JoinType) → "Join": Sets the join_type attribute.

to_dict() → dict: Returns the Join as a dictionary.

CONST_JOIN_TYPE = 'joinType'

CONST_LEFT_ON = 'leftOn'

CONST_ON = 'on'

CONST_QUERY = 'query'

CONST_RIGHT_ON = 'rightOn'

property join_type: list

property left_on: list

property on: List[str]

property right_on: list

property sub_query: Query

to_dict()

Returns the Join as dictionary.

with_join_type(join_type: JoinType) → Join

Sets the left feature details.

Parameters:

join_type (JoinType) – The join type: Defaults to INNER

Returns:

The Join instance (self)

Return type:

Join

with_left_on(left_on: List[str]) → Join

Sets the left feature details.

Parameters:

left_on (List[str]) – The left feature to join on

Returns:

The Join instance (self)

Return type:

Join

with_on(on: List[str] = [])

Sets the query.

Parameters:

on (List[str]) – The on clause .

Returns:

The Join instance (self)

Return type:

Join

with_right_on(right_on: List[str]) → Join

Sets the left feature details.

Parameters:

right_on (List[str]) – The right feature to join on

Returns:

The Join instance (self)

Return type:

Join

with_sub_query(sub_query: Query) → Join

Sets the query.

Parameters:

sub_query (Query) – The query .

Returns:

The Join instance (self)

Return type:

Join

ads.feature_store package

class ads.feature_store.dataset_job.DatasetJob(spec: Dict | None = None, **kwargs)

Bases: Builder

Represents an DatasetJob Resource.

create(self, \*\*kwargs) → 'DatasetJob'

Creates dataset_run resource.

from_id(cls, id: str) → 'DatasetJob'

Gets an existing dataset_run resource by id.

list(cls, compartment_id: str = None, \*\*kwargs) → List['DatasetJob']

Lists dataset_run resources in a given compartment.

list_df(cls, compartment_id: str = None, \*\*kwargs) → 'pandas.DataFrame'

Lists dataset_run resources as a pandas dataframe.

with_description(self, description: str) → 'DatasetJob'

Sets the description.

with_compartment_id(self, compartment_id: str) → 'DatasetJob'

Sets the compartment ID.

with_dataset_id(self, dataset_id: str) → 'DatasetJob'

Sets the dataset ID.

with_display_name(self, name: str) → 'DatasetJob'

Sets the name.

with_ingestion_mode(self, ingestion_mode: IngestionMode) → 'DatasetJob'

Sets the ingestion mode.

Examples

>>> from ads.feature_store import dataset_job
>>> import oci
>>> import os
>>> dataset_run = dataset_run.DatasetJob()
>>>     .with_compartment_id(os.environ["PROJECT_COMPARTMENT_OCID"])
>>>     .with_dataset_id("dataset_id")
>>>     .with_ingestion_mode(BatchIngestionMode.SQL)
>>> dataset_run.create()

CONST_COMPARTMENT_ID = 'compartmentId'

CONST_DATASET_ID = 'datasetId'

CONST_DATA_FLOW_EXECUTION_OUTPUT = 'dataFlowBatchExecutionOutput'

CONST_DATA_FLOW_READ_WRITE_DETAIL = 'data_flow_read_write_detail'

CONST_DATA_READ_IN_BYTES = 'data_read_in_bytes'

CONST_DATA_WRITTEN_BYTES = 'data_written_in_bytes'

CONST_DEFINED_TAG = 'definedTags'

CONST_ERROR_DETAILS = 'errorDetails'

CONST_FEATURE_OPTION_DETAILS = 'featureOptionsDetails'

CONST_FREEFORM_TAG = 'freeformTags'

CONST_ID = 'id'

CONST_INGESTION_MODE = 'ingestionMode'

CONST_JOB_CONFIGURATION_DETAILS = 'jobConfigurationDetails'

CONST_JOB_CONFIGURATION_TYPE = 'jobConfigurationType'

CONST_JOB_OUTPUT_DETAILS = 'jobOutputDetails'

CONST_LIFECYCLE_STATE = 'lifecycleState'

CONST_TIME_FROM = 'timeFrom'

CONST_TIME_TO = 'timeTo'

CONST_VALIDATION_OUTPUT = 'validation_output'

attribute_map = {'compartmentId': 'compartment_id', 'datasetId': 'dataset_id', 'definedTags': 'defined_tags', 'featureOptionsDetails': 'feature_option_details', 'freeformTags': 'freeform_tags', 'id': 'id', 'ingestionMode': 'ingestion_mode', 'jobConfigurationDetails': 'job_configuration_details', 'jobConfigurationType': 'job_configuration_type', 'jobOutputDetails': 'job_output_details', 'timeFrom': 'time_from', 'timeTo': 'time_to'}

property compartment_id: str

create(**kwargs) → DatasetJob

Creates dataset_run resource.

Parameters:

kwargs – Additional kwargs arguments. Can be any attribute that feature_store.models.DatasetJob accepts.

Returns:

The DatasetJob instance (self)

Return type:

FeatureStore

Raises:

ValueError – If compartment id not provided.

property dataset_id: str

property feature_option_details

classmethod from_id(id: str) → DatasetJob

Gets an existing dataset_run resource by Id.

Parameters:

id (str) – The dataset_run id.

Returns:

An instance of DatasetJob resource.

Return type:

FeatureStore

property id: str

property ingestion_mode: str

property job_configuration_details: str

property job_output_details: Dict

property kind: str

The kind of the object as showing in a YAML.

classmethod list(compartment_id: str | None = None, **kwargs) → List[DatasetJob]

Lists DatasetJob Resources in a given compartment.

Parameters:

• compartment_id ((str, optional). Defaults to None.) – The compartment OCID.

• kwargs – Additional keyword arguments for filtering DatasetJob.

Returns:

The list of the DatasetJob Resources.

Return type:

List[DatasetJob]

classmethod list_df(compartment_id: str | None = None, **kwargs) → DataFrame

Lists dataset_run resources in a given compartment.

Parameters:

• compartment_id ((str, optional). Defaults to None.) – The compartment OCID.

• kwargs – Additional keyword arguments for filtering models.

Returns:

The list of the dataset_run resources in a pandas dataframe format.

Return type:

pandas.DataFrame

to_dict() → Dict

Serializes dataset_run to a dictionary.

Returns:

The dataset_run resource serialized as a dictionary.

Return type:

dict

update(**kwargs) → DatasetJob

Updates DatasetJob in the feature store.

Parameters:

kwargs – Additional kwargs arguments. Can be any attribute that feature_store.models.DatasetJob accepts.

Returns:

The DatasetJob instance (self).

Return type:

DatasetJob

with_compartment_id(compartment_id: str) → DatasetJob

Sets the compartment_id.

Parameters:

compartment_id (str) – The compartment_id.

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

with_dataset_id(dataset_id: str) → DatasetJob

Sets the dataset_id.

Parameters:

dataset_id (str) – The dataset id.

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

with_error_details(error_details: str) → DatasetJob

Sets the error details.

Parameters:

error_details (str) – The error_details.

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

with_feature_option_details(feature_option_details: FeatureOptionDetails) → DatasetJob

Sets the feature_option_details.

Parameters:

feature_option_details (FeatureOptionDetails) –

Returns:

The FeatureGroupJob instance (self)

Return type:

DatasetJob

with_id(id: str) → DatasetJob

with_ingestion_mode(ingestion_mode: BatchIngestionMode | StreamingIngestionMode) → DatasetJob

Sets the mode of the dataset ingestion mode.

Parameters:

ingestion_mode (IngestionMode) – The mode of the dataset ingestion mode.

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

with_job_configuration_details(job_configuration_type: JobConfigurationType, **kwargs: Dict[str, Any]) → DatasetJob

Sets the job configuration details.

Parameters:

• job_configuration_type (JobConfigurationType) – The job_configuration_type of job

• kwargs (Dict[str, Any]) – Additional key value arguments

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

with_job_output_details(job_output_details: Dict) → DatasetJob

Sets the job output details.

Parameters:

job_output_details (Dict) – The job output details which contains error_details, validation_output and commit id.

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

with_lifecycle_state(lifecycle_state: str) → DatasetJob

Sets the lifecycle_state.

Parameters:

lifecycle_state (str) – The lifecycle_state.

Returns:

The DatasetJob instance (self)

Return type:

DatasetJob

class ads.feature_store.feature.DatasetFeature(name, featureType, datasetId)

Bases: Builder

A class that represents a feature and allows comparison with other features using various operators. The Feature class has six comparison methods (__lt__, __le__, __eq__, __ne__, __ge__, and __gt__) that return instances of a Filter class. These comparison methods allow instances of the Feature class to be compared using the corresponding comparison operators.

CONST_DATASET_ID = 'datasetId'

CONST_FEATURE_NAME = 'name'

CONST_FEATURE_TYPE = 'featureType'

property dataset_id

property feature_name

property feature_type

to_dict() → Dict

Serializes feature to a dictionary.

Returns:

The feature serialized as a dictionary.

Return type:

dict

with_dataset_id(dataset_id)

Sets the group attribute of the feature.

Parameters:

dataset_id – Dataset id which contains the feature.

Returns:

This instance of the Feature class.

Return type:

Feature

with_feature_name(name: str)

Sets the name attribute of the feature.

Parameters:

name (str) – The new name for the feature.

Returns:

This instance of the Feature class.

Return type:

Feature

with_feature_type(feature_type: str)

Sets the type attribute of the feature.

Parameters:

feature_type (str) – The new type for the feature.

Returns:

This instance of the Feature class.

Return type:

Feature

class ads.feature_store.feature.Feature(name, featureType, featureGroupId)

Bases: Builder

A class that represents a feature and allows comparison with other features using various operators. The Feature class has six comparison methods (__lt__, __le__, __eq__, __ne__, __ge__, and __gt__) that return instances of a Filter class. These comparison methods allow instances of the Feature class to be compared using the corresponding comparison operators.

CONST_FEATURE_GROUP_ID = 'featureGroupId'

CONST_FEATURE_NAME = 'name'

CONST_FEATURE_TYPE = 'featureType'

property feature_group_id

property feature_name

property feature_type

to_dict() → Dict

Serializes feature to a dictionary.

Returns:

The feature serialized as a dictionary.

Return type:

dict

with_feature_group_id(feature_group_id)

Sets the group attribute of the feature.

Parameters:

feature_group_id – FeatureGroup id which contains the feature.

Returns:

This instance of the Feature class.

Return type:

Feature

with_feature_name(name: str)

Sets the name attribute of the feature.

Parameters:

name (str) – The new name for the feature.

Returns:

This instance of the Feature class.

Return type:

Feature

with_feature_type(feature_type: str)

Sets the type attribute of the feature.

Parameters:

feature_type (str) – The new type for the feature.

Returns:

This instance of the Feature class.

Return type:

Feature

class ads.feature_store.feature_group_job.FeatureGroupJob(spec: Dict | None = None, **kwargs)

Bases: Builder

Represents an FeatureGroupJob Resource.

create(self, \*\*kwargs) → 'FeatureGroupJob'

Creates feature_group_run resource.

from_id(cls, id: str) → 'FeatureGroupJob'

Gets an existing feature_group_run resource by id.

list(cls, compartment_id: str = None, \*\*kwargs) → List['FeatureGroupJob']

Lists feature_group_run resources in a given compartment.

list_df(cls, compartment_id: str = None, \*\*kwargs) → 'pandas.DataFrame'

Lists feature_group_run resources as a pandas dataframe.

with_compartment_id(self, compartment_id: str) → 'FeatureGroupJob'

Sets the compartment ID.

with_feature_group_id(self, feature_group_id: str) → 'FeatureGroupJob'

Sets the feature group ID.

with_ingestion_mode(self, ingestion_mode: IngestionMode) → 'FeatureGroupJob'

Sets the ingestion mode.

Examples

>>> from ads.feature_store import feature_group_job
>>> import oci
>>> import os
>>> feature_group_run = feature_group_run.FeatureGroupJob()
>>>     .with_compartment_id(os.environ["PROJECT_COMPARTMENT_OCID"])
>>>     .with_feature_group_id("<feature_group_id>")
>>>     .with_ingestion_mode(IngestionMode.OVERWRITE)
>>> feature_group_run.create()

CONST_COMPARTMENT_ID = 'compartmentId'

CONST_DATA_FLOW_EXECUTION_OUTPUT = 'dataFlowBatchExecutionOutput'

CONST_DATA_FLOW_READ_WRITE_DETAIL = 'data_flow_read_write_detail'

CONST_DATA_READ_IN_BYTES = 'data_read_in_bytes'

CONST_DATA_WRITTEN_BYTES = 'data_written_in_bytes'

CONST_DEFINED_TAG = 'definedTags'

CONST_FEATURE_GROUP_ID = 'featureGroupId'

CONST_FEATURE_OPTION_DETAILS = 'featureOptionsDetails'

CONST_FEATURE_STATISTICS = 'featureStatistics'

CONST_FREEFORM_TAG = 'freeformTags'

CONST_ID = 'id'

CONST_INGESTION_MODE = 'ingestionMode'

CONST_JOB_CONFIGURATION_DETAILS = 'jobConfigurationDetails'

CONST_JOB_CONFIGURATION_TYPE = 'jobConfigurationType'

CONST_JOB_OUTPUT_DETAILS = 'jobOutputDetails'

CONST_LIFECYCLE_STATE = 'lifecycleState'

CONST_TIME_FROM = 'timeFrom'

CONST_TIME_TO = 'timeTo'

CONST_VALIDATION_OUTPUT = 'validation_output'

attribute_map = {'compartmentId': 'compartment_id', 'dataFlowBatchExecutionOutput': 'data_flow_batch_execution_output', 'definedTags': 'defined_tags', 'featureGroupId': 'feature_group_id', 'featureOptionsDetails': 'feature_option_details', 'featureStatistics': 'feature_statistics', 'freeformTags': 'freeform_tags', 'id': 'id', 'ingestionMode': 'ingestion_mode', 'jobConfigurationDetails': 'job_configuration_details', 'jobConfigurationType': 'job_configuration_type', 'jobOutputDetails': 'job_output_details', 'lifecycleState': 'lifecycle_state', 'timeFrom': 'time_from', 'timeTo': 'time_to'}

property compartment_id: str

create(**kwargs) → FeatureGroupJob

Creates feature_group_run resource.

Parameters:

kwargs – Additional kwargs arguments. Can be any attribute that feature_store.models.FeatureGroupJob accepts.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureStore

Raises:

ValueError – If compartment id not provided.

property feature_group_id: str

property feature_option_details: Dict

property feature_statistics: str

classmethod from_id(id: str) → FeatureGroupJob

Gets an existing feature_group_run resource by Id.

Parameters:

id (str) – The feature_group_run id.

Returns:

An instance of FeatureGroupJob resource.

Return type:

FeatureStore

get_validation_output_df() → DataFrame

This method retrieves the validation output as a Pandas DataFrame.

Returns: pandas.DataFrame – The validation output data in DataFrame format.

property id: str

property ingestion_mode: str

property job_configuration_details: str

property job_output_details: Dict

property kind: str

The kind of the object as showing in a YAML.

classmethod list(compartment_id: str | None = None, **kwargs) → List[FeatureGroupJob]

Lists FeatureGroupJob Resources in a given compartment.

Parameters:

• compartment_id ((str, optional). Defaults to None.) – The compartment OCID.

• kwargs – Additional keyword arguments for filtering FeatureGroupJob.

Returns:

The list of the FeatureGroupJob Resources.

Return type:

List[FeatureGroupJob]

classmethod list_df(compartment_id: str | None = None, **kwargs) → DataFrame

Lists feature_group_run resources in a given compartment.

Parameters:

• compartment_id ((str, optional). Defaults to None.) – The compartment OCID.

• kwargs – Additional keyword arguments for filtering models.

Returns:

The list of the feature_group_run resources in a pandas dataframe format.

Return type:

pandas.DataFrame

property time_from: str

property time_to: str

to_dict() → Dict

Serializes feature_group_run to a dictionary.

Returns:

The feature_group_run resource serialized as a dictionary.

Return type:

dict

update(**kwargs) → FeatureGroupJob

Updates FeatureGroupJob in the feature store.

Parameters:

kwargs – Additional kwargs arguments. Can be any attribute that feature_store.models.FeatureGroupJob accepts.

Returns:

The FeatureGroupJob instance (self).

Return type:

FeatureGroupJob

with_compartment_id(compartment_id: str) → FeatureGroupJob

Sets the compartment_id.

Parameters:

compartment_id (str) – The compartment_id.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_feature_group_id(feature_group_id: str) → FeatureGroupJob

Sets the feature_group_id.

Parameters:

feature_group_id (str) – The feature group id.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_feature_option_details(feature_option_details: FeatureOptionDetails) → FeatureGroupJob

Sets the feature_option_details.

Parameters:

feature_option_details (FeatureOptionDetails) –

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_feature_statistics(feature_statistics: str) → FeatureGroupJob

Sets the computed statistics.

Parameters:

feature_statistics (str) – Computed Feature Statistics

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_id(id: str) → FeatureGroupJob

with_ingestion_mode(ingestion_mode: BatchIngestionMode | StreamingIngestionMode) → FeatureGroupJob

Sets the mode of the dataset ingestion mode.

Parameters:

ingestion_mode –

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_job_configuration_details(job_configuration_type: JobConfigurationType, **kwargs: Dict[str, Any]) → FeatureGroupJob

Sets the job configuration details.

Parameters:

• job_configuration_type (JobConfigurationType) – The job_configuration_type of job

• kwargs (Dict[str, Any]) – Additional key value arguments

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_job_output_details(job_output_details: Dict) → FeatureGroupJob

Sets the job output details.

Parameters:

job_output_details (Dict) – The job output details which contains error_details, validation_output and commit id.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_lifecycle_state(lifecycle_state: str) → FeatureGroupJob

Sets the lifecycle_state.

Parameters:

lifecycle_state (str) – The lifecycle_state.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_time_from(time_from: str) → FeatureGroupJob

Sets the time_from.

Parameters:

time_from (str) – The time_from.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

with_time_to(time_to: str) → FeatureGroupJob

Sets the time_to.

Parameters:

time_to (str) – The time_to.

Returns:

The FeatureGroupJob instance (self)

Return type:

FeatureGroupJob

class ads.feature_store.feature_group_job.JobConfigurationType(value)

Bases: Enum

An enumeration.

SPARK_BATCH_AUTOMATIC = 'SPARK_BATCH_AUTOMATIC'

SPARK_BATCH_MANUAL = 'SPARK_BATCH_MANUAL'

class ads.feature_store.feature_option_details.FeatureOptionDetails

Bases: Builder

CONST_FEATURE_OPTION_WRITE_CONFIG_DETAILS = 'featureOptionWriteConfigDetails'

CONST_MERGE_SCHEMA = 'mergeSchema'

CONST_OVERWRITE_SCHEMA = 'overwriteSchema'

to_dict()

Returns the FeatureOptionDetails as dictionary.

with_feature_option_write_config_details(merge_schema: bool = False, overwrite_schema: bool = False) → FeatureOptionDetails

Sets the feature option write configuration details.

Parameters:

• merge_schema (bool) – The merge_schema.

• overwrite_schema (bool) – The overwrite_schema.

Returns:

The FeatureOptionDetails instance (self)

Return type:

FeatureOptionDetails

class ads.feature_store.input_feature_detail.FeatureDetail(name: str, feature_type: FeatureType | None = None, order_number: int | None = None, is_event_timestamp: bool = False, event_timestamp_format: str | None = None)

Bases: Builder

Represents input Feature Schema.

with_feature_type(self, feature_type: FeatureType) → 'FeatureDetail'

Sets the feature_type.

with_order_number(self, order_number: int) → 'FeatureDetail'

Sets the order_number.

with_event_timestamp_format(self, event_timestamp_format: str) → 'FeatureDetail'

Sets the timestamp format for the feature.

with_is_event_timestamp_format(self, is_event_timestamp_format: bool) → 'FeatureDetail'

Sets the is_event_timestamp_format.

CONST_EVENT_TIMESTAMP_FORMAT = 'eventTimestampFormat'

CONST_FEATURE_TYPE = 'featureType'

CONST_IS_EVENT_TIMESTAMP = 'isEventTimestamp'

CONST_NAME = 'name'

CONST_ORDER_NUMBER = 'orderNumber'

property event_timestamp_format

property feature_name

property feature_type

property is_event_timestamp

property order_number

to_dict()

Returns the FeatureDetail as dictionary.

with_event_timestamp_format(event_timestamp_format: str) → FeatureDetail

Sets the event_timestamp_format.

Parameters:

event_timestamp_format (str) – The event_timestamp_format of the Feature.

Returns:

The FeatureDetail instance (self)

Return type:

FeatureDetail

with_feature_type(feature_type: FeatureType) → FeatureDetail

Sets the feature_type.

Parameters:

feature_type (FeatureType) – The feature_type of the Feature.

Returns:

The FeatureDetails instance (self)

Return type:

FeatureDetails

with_is_event_timestamp(is_event_timestamp: bool) → FeatureDetail

Sets the is_event_timestamp.

Parameters:

is_event_timestamp (bool) – The is_event_timestamp of the Feature.

Returns:

The FeatureDetail instance (self)

Return type:

FeatureDetail

with_order_number(order_number: int) → FeatureDetail

Sets the order number.

Parameters:

order_number (int) – The order_number of the Feature.

Returns:

The FeatureDetail instance (self)

Return type:

FeatureDetail

class ads.feature_store.model_details.ModelDetails(items: List[str] | None = None)

Bases: Builder

Sets the model Details. .. method:: with_items(self, items: List[str]) -> “ModelDetails”

Set the model ids associated with a dataset

CONST_ITEMS = 'items'

attribute_map = {'items': 'items'}

property items: List[str]

property kind: str

The kind of the object as showing in a YAML.

to_dict() → Dict

Serializes rule to a dictionary.

Returns:

The rule resource serialized as a dictionary.

Return type:

dict

with_items(items: List[str]) → ModelDetails

Sets the model ids associated with dataset.

Parameters:

items (List[str]) – items array of model ids

Returns:

The ModelDetails instance (self)

Return type:

ModelDetails

class ads.feature_store.statistics_config.StatisticsConfig(is_enabled: bool = True, columns: List[str] | None = None)

Bases: Builder

Sets the Statistics Config Details. .. method:: with_enabled(self, enabled: bool) -> “StatisticsConfig”

Sets True/False for enabled

with_columns(self, columns: List[str]) → 'StatisticsConfig'

Sets the column names for the statistics config

CONST_COLUMNS = 'columns'

CONST_ENABLED = 'isEnabled'

attribute_map = {'columns': 'columns', 'isEnabled': 'is_enabled'}

property columns: List[str]

property is_enabled: bool

property kind: str

The kind of the object as showing in a YAML.

to_dict() → Dict

Serializes rule to a dictionary.

Returns:

The rule resource serialized as a dictionary.

Return type:

dict

with_columns(columns: List[str]) → StatisticsConfig

Sets the columns for which the stats to be calculated .

Parameters:

columns (List[str]) – columns for which the stats to be calculated.

Returns:

The StatisticsConfig instance (self)

Return type:

StatisticsConfig

with_is_enabled(is_enabled: bool) → StatisticsConfig

Sets True/False for enabled

Parameters:

is_enabled (bool) – enable or disable the statistics computation

Returns:

The StatisticsConfig instance (self)

Return type:

StatisticsConfig

class ads.feature_store.transformation.Transformation(spec: Dict | None = None, **kwargs)

Bases: Builder

Represents a Transformation Resource.

create(self, \*\*kwargs) → 'Transformation'

Creates transformation resource.

delete(self) → "Transformation":

Removes transformation resource.

to_dict(self) → dict

Serializes transformation to a dictionary.

from_id(cls, id: str) → 'Transformation'

Gets an existing transformation resource by id.

list(cls, compartment_id: str = None, \*\*kwargs) → List['Transformation']

Lists transformation resources in a given compartment.

list_df(cls, compartment_id: str = None, \*\*kwargs) → 'pandas.DataFrame'

Lists transformation resources as a pandas dataframe.

with_description(self, description: str) → 'Transformation'

Sets the description.

with_compartment_id(self, compartment_id: str) → 'Transformation'

Sets the compartment ID.

with_feature_store_id(self, feature_store_id: str) → 'Transformation'

Sets the feature store ID.

with_name(self, name: str) → 'Transformation'

Sets the name.

with_transformation_mode(self, transformation_mode: TransformationMode) → 'Transformation'

Sets the transformation mode.

with_source_code_function(self, source_code_func) → 'Transformation'

Sets the transformation source code function.

Examples

>>> from ads.feature_store import transformation
>>> import oci
>>> import os
>>> def transactions_df(transactions_batch):
>>>        sql_query = f"select id, cc_num, amount from {transactions_batch}"
>>>        return sql_query
>>>
>>> transformation = transformation.Transformation()
>>>     .with_description("Feature store description")
>>>     .with_compartment_id(os.environ["PROJECT_COMPARTMENT_OCID"])
>>>     .with_name("FeatureStore")
>>>     .with_feature_store_id("feature_store_id")
>>>     .with_transformation_mode(TransformationMode.SQL)
>>>     .with_source_code_function(transactions_df)
>>> transformation.create()

CONST_COMPARTMENT_ID = 'compartmentId'

CONST_DEFINED_TAG = 'definedTags'

CONST_DESCRIPTION = 'description'

CONST_FEATURE_STORE_ID = 'featureStoreId'

CONST_FREEFORM_TAG = 'freeformTags'

CONST_FUNCTION_REF = 'functionRef'

CONST_ID = 'id'

CONST_NAME = 'name'

CONST_SOURCE_CODE = 'sourceCode'

CONST_TRANSFORMATION_MODE = 'transformationMode'

attribute_map = {'compartmentId': 'compartment_id', 'definedTags': 'defined_tags', 'description': 'description', 'featureStoreId': 'feature_store_id', 'freeformTags': 'freeform_tags', 'id': 'id', 'name': 'name', 'sourceCode': 'source_code', 'transformationMode': 'transformation_mode'}

property compartment_id: str

create(**kwargs) → Transformation

Creates transformation resource.

Parameters:

kwargs – Additional kwargs arguments. Can be any attribute that feature_store.models.Transformation accepts.

Returns:

The Transformation instance (self)

Return type:

FeatureStore

Raises:

ValueError – If compartment id not provided.

delete()

Removes transformation resource.

Return type:

None

property description: str

property feature_store_id: str

classmethod from_id(id: str) → Transformation

Gets an existing Transformation resource by Id.

Parameters:

id (str) – The Transformation id.

Returns:

An instance of Transformation resource.

Return type:

FeatureStore

property id: str

property kind: str

The kind of the object as showing in a YAML.

classmethod list(compartment_id: str | None = None, **kwargs) → List[Transformation]

Lists transformation resources in a given compartment.

Parameters:

• compartment_id ((str, optional). Defaults to None.) – The compartment OCID.

• kwargs – Additional keyword arguments for filtering Transformation.

Returns:

The list of the Transformation Resources.

Return type:

List[Transformation]

classmethod list_df(compartment_id: str | None = None, **kwargs) → DataFrame

Lists transformation resources in a given compartment.

Parameters:

• compartment_id ((str, optional). Defaults to None.) – The compartment OCID.

• kwargs – Additional keyword arguments for filtering models.

Returns:

The list of the transformation resources in a pandas dataframe format.

Return type:

pandas.DataFrame

property name: str

property source_code_function: str

to_dict() → Dict

Serializes transformation to a dictionary.

Returns:

The Transformation resource serialized as a dictionary.

Return type:

dict

property transformation_mode: str

with_compartment_id(compartment_id: str) → Transformation

Sets the compartment_id.

Parameters:

compartment_id (str) – The compartment_id.

Returns:

The Transformation instance (self)

Return type:

Transformation

with_description(description: str) → Transformation

Sets the description.

Parameters:

description (str) – The description of the transformation resource.

Returns:

The Transformation instance (self)

Return type:

FeatureStore

with_feature_store_id(feature_store_id: str) → Transformation

Sets the feature_store_id.

Parameters:

feature_store_id (str) – The featurestore id.

Returns:

The Transformation instance (self)

Return type:

Transformation

with_id(id: str) → Transformation

with_name(name: str) → Transformation

Sets the name.

Parameters:

name (str) – The name of Transformation resource.

Returns:

The Transformation instance (self)

Return type:

Transformation

with_source_code_function(source_code_func) → Transformation

Sets the source code function for the transformation.

Parameters:

source_code_func (function) – source code for the transformation.

Returns:

The Transformation instance (self)

Return type:

Transformation

with_transformation_mode(transformation_mode: TransformationMode) → Transformation

Sets the mode of the transformation.

Parameters:

transformation_mode (TransformationMode) – The mode of the transformation.

Returns:

The Transformation instance (self)

Return type:

Transformation

class ads.feature_store.validation_output.ValidationOutput(content: str, version: int = 1)

Bases: ResponseBuilder

Represents validation output results class after validation.

property kind: str

Gets the kind of the validation output object.

Returns:

The kind of the validation output object, which is always “ValidationOutput”.

Return type:

str

to_pandas() → DataFrame

Converts the validation output information to a pandas DataFrame.

Returns:

The validation output information as a pandas DataFrame.

Return type:

pd.DataFrame

to_summary() → DataFrame

Converts the validation output summary information to a pandas DataFrame.

Returns:

The validation output summary information as a pandas DataFrame.

Return type:

pd.DataFrame

ads.feature_store.statistics package

ads.feature_store.common package

class ads.feature_store.common.enums.BatchIngestionMode(value)

Bases: Enum

An enumeration that represents the supported Ingestion Mode in feature store.

OVERWRITE

Ingestion mode to overwrite the data in the system.

Type:

str

APPEND

Ingestion mode to append the data in the system.

Type:

str

UPSERT

Ingestion mode to insert and update the data in the system.

Type:

str

None()

APPEND = 'APPEND'

DEFAULT = 'DEFAULT'

OVERWRITE = 'OVERWRITE'

UPSERT = 'UPSERT'

class ads.feature_store.common.enums.DataFrameType(value)

Bases: Enum

An enumeration that represents the supported DataFrame types.

SPARK

A string representation for spark Data frame type.

Type:

str

PANDAS

A string representation for pandas Data frame type.

Type:

str

None()

PANDAS = 'PANDAS'

SPARK = 'SPARK'

class ads.feature_store.common.enums.DatasetIngestionMode(value)

Bases: Enum

An enumeration defining the possible modes for ingesting datasets.

SQL

A string representing the SQL mode, which is used to ingest datasets using SQL.

Type:

str

SQL = 'SQL'

class ads.feature_store.common.enums.EntityType(value)

Bases: Enum

An enumeration of the supported entity types.

FEATURE_GROUP

A string representing the feature group.

Type:

str

DATASET

An string representing the dataset.

Type:

str

DATASET = 'DATASET'

FEATURE_GROUP = 'FEATURE_GROUP'

class ads.feature_store.common.enums.ExecutionEngine(value)

Bases: Enum

An enumeration that represents the supported execution engines.

SPARK

A string representation of the Apache Spark execution engine.

Type:

str

PANDAS

A string representation of the Pandas execution engine.

Type:

str

None()

PANDAS = 'PANDAS'

SPARK = 'SPARK'

class ads.feature_store.common.enums.ExpectationType(value)

Bases: Enum

An enumeration of the available expectation types for a feature store.

STRICT

A strict expectation type.

Type:

str

LENIENT

A lenient expectation type.

Type:

str

NO_EXPECTATION

A no expectation type.

Type:

str

None()

LENIENT = 'LENIENT'

NO_EXPECTATION = 'NO_EXPECTATION'

STRICT = 'STRICT'

class ads.feature_store.common.enums.FeatureStoreJobType(value)

Bases: Enum

An enumeration that represents the Job type.

FEATURE_GROUP_INGESTION

A string representing that job is feature group ingestion.

Type:

str

DATASET_INGESTION

A string representing that job is dataset ingestion.

Type:

str

FEATURE_GROUP_DELETION

A string representing that job is feature group deletion.

Type:

str

DATASET_DELETION

A string representing that job is dataset deletion.

Type:

str

None()

DATASET_DELETION = 'DATASET_DELETION'

DATASET_INGESTION = 'DATASET_INGESTION'

FEATURE_GROUP_DELETION = 'FEATURE_GROUP_DELETION'

FEATURE_GROUP_INGESTION = 'FEATURE_GROUP_INGESTION'

class ads.feature_store.common.enums.FeatureType(value)

Bases: Enum

An enumeration of the available feature types for a feature store.

STRING

A string feature type.

Type:

str

INTEGER

An integer feature type.

Type:

str

FLOAT

A float feature type.

Type:

str

DOUBLE

A double feature type.

Type:

str

BOOLEAN

A boolean feature type.

Type:

str

DATE

A date feature type.

Type:

str

TIMESTAMP

A timestamp feature type.

Type:

str

DECIMAL

A decimal feature type.

Type:

str

BINARY

A binary feature type.

Type:

str

ARRAY

An array feature type.

Type:

str

MAP

A map feature type.

Type:

str

STRUCT

A struct feature type.

Type:

str

BINARY = 'BINARY'

BINARY_ARRAY = 'BINARY_ARRAY'

BOOLEAN = 'BOOLEAN'

BOOLEAN_ARRAY = 'BOOLEAN_ARRAY'

BYTE = 'BYTE'

BYTE_ARRAY = 'BYTE_ARRAY'

COMPLEX = 'COMPLEX'

DATE = 'DATE'

DATE_ARRAY = 'DATE_ARRAY'

DECIMAL = 'DECIMAL'

DOUBLE = 'DOUBLE'

DOUBLE_ARRAY = 'DOUBLE_ARRAY'

FLOAT = 'FLOAT'

FLOAT_ARRAY = 'FLOAT_ARRAY'

INTEGER = 'INTEGER'

INTEGER_ARRAY = 'INTEGER_ARRAY'

LONG = 'LONG'

LONG_ARRAY = 'LONG_ARRAY'

SHORT = 'SHORT'

SHORT_ARRAY = 'SHORT_ARRAY'

STRING = 'STRING'

STRING_ARRAY = 'STRING_ARRAY'

STRING_BINARY_MAP = 'STRING_BINARY_MAP'

STRING_BOOLEAN_MAP = 'STRING_BOOLEAN_MAP'

STRING_BYTE_MAP = 'STRING_BYTE_MAP'

STRING_DATE_MAP = 'STRING_DATE_MAP'

STRING_DOUBLE_MAP = 'STRING_DOUBLE_MAP'

STRING_FLOAT_MAP = 'STRING_FLOAT_MAP'

STRING_INTEGER_MAP = 'STRING_INTEGER_MAP'

STRING_LONG_MAP = 'STRING_LONG_MAP'

STRING_SHORT_MAP = 'STRING_SHORT_MAP'

STRING_STRING_MAP = 'STRING_STRING_MAP'

STRING_TIMESTAMP_MAP = 'STRING_TIMESTAMP_MAP'

TIMESTAMP = 'TIMESTAMP'

TIMESTAMP_ARRAY = 'TIMESTAMP_ARRAY'

UNKNOWN = 'UNKNOWN'

class ads.feature_store.common.enums.FilterOperators(value)

Bases: Enum

An enumeration defining the different comparison operators for data filtering.

GE

A string representing the greater than or equal to operator.

Type:

str

GT

A string representing the greater than operator.

Type:

str

NE

A string representing the not equals operator.

Type:

str

EQ

A string representing the equals operator.

Type:

str

LE

A string representing the less than or equal to operator.

Type:

str

LT

A string representing the less than operator.

Type:

str

IN

A string representing the in operator.

Type:

str

LK

A string representing the like operator.

Type:

str

EQ = 'EQUALS'

GE = 'GREATER_THAN_OR_EQUAL'

GT = 'GREATER_THAN'

IN = 'IN'

LE = 'LESS_THAN_OR_EQUAL'

LK = 'LIKE'

LT = 'LESS_THAN'

NE = 'NOT_EQUALS'

class ads.feature_store.common.enums.IngestionType(value)

Bases: Enum

The type of ingestion that can be performed.

Possible values:

• STREAMING: The data is ingested in real time.

• BATCH: The data is ingested in batches.

BATCH = 'BATCH'

STREAMING = 'STREAMING'

class ads.feature_store.common.enums.JobConfigurationType(value)

Bases: Enum

An enumeration defining the different types of job configuration modes for Spark.

SPARK_BATCH_AUTOMATIC

A string representing automatic job configuration mode for Spark Batch jobs.

Type:

str

SPARK_BATCH_MANUAL

A string representing manual job configuration mode for Spark Batch jobs.

Type:

str

SPARK_BATCH_AUTOMATIC = 'SPARK_BATCH_AUTOMATIC'

SPARK_BATCH_MANUAL = 'SPARK_BATCH_MANUAL'

class ads.feature_store.common.enums.JobStatus(value)

Bases: Enum

An enumeration that represents the supported Job status.

SUCCEEDED

A string representation of the state of Succeeded job.

Type:

str

FAILED

A string representation of the state of Failed job.

Type:

str

CODE_EXECUTION

A string representation of the state of CodeExecution job.

Type:

str

None()

CODE_EXECUTION = 'CODE_EXECUTION'

FAILED = 'FAILED'

SUCCEEDED = 'SUCCEEDED'

class ads.feature_store.common.enums.JoinType(value)

Bases: Enum

Enumeration of supported SQL join types.

INNER

Inner join.

LEFT

Left join.

RIGHT

Right join.

FULL

Full outer join.

CROSS

Cross join.

LEFT_SEMI_JOIN

Left semi join.

CROSS = 'CROSS'

FULL = 'FULL'

INNER = 'INNER'

LEFT = 'LEFT'

LEFT_SEMI_JOIN = 'LEFT_SEMI_JOIN'

RIGHT = 'RIGHT'

class ads.feature_store.common.enums.LevelType(value)

Bases: Enum

An enumeration defining the different types of logging levels.

ERROR

A string representing the highest logging level, indicating an error in the program.

Type:

str

WARNING

A string representing a lower logging level, indicating a potential issue or warning in the program.

Type:

str

ERROR = 'ERROR'

WARNING = 'WARNING'

class ads.feature_store.common.enums.LifecycleState(value)

Bases: Enum

An enumeration that represents the lifecycle state of feature store resources.

ACTIVE

A string representing Active resource.

Type:

str

FAILED

A string representing Failed resource.

Type:

str

NEEDS_ATTENTION

A string representing needs_attention resource.

Type:

str

None()

ACTIVE = 'ACTIVE'

FAILED = 'FAILED'

NEEDS_ATTENTION = 'NEEDS_ATTENTION'

class ads.feature_store.common.enums.StreamingIngestionMode(value)

Bases: Enum

Enumeration for stream ingestion modes.

• COMPLETE: Represents complete stream ingestion where the entire dataset is replaced.

• APPEND: Represents appending new data to the existing dataset.

• UPDATE: Represents updating existing data in the dataset.

APPEND = 'APPEND'

COMPLETE = 'COMPLETE'

UPDATE = 'UPDATE'

class ads.feature_store.common.enums.TransformationMode(value)

Bases: Enum

An enumeration defining the different modes for data transformation.

SQL

A string representing the SQL mode, which is used to transform data using SQL queries.

Type:

str

PANDAS

A string representing the Pandas mode, which is used to transform data using the Pandas library.

Type:

str

PANDAS = 'pandas'

SPARK = 'spark'

SQL = 'sql'

class ads.feature_store.common.enums.ValidationEngineType(value)

Bases: Enum

An enumeration that represents the supported validation engines.

GREAT_EXPECTATIONS

A string representation of the great expectation execution engine.

Type:

str

None()

GREAT_EXPECTATIONS = 'GREAT_EXPECTATIONS'