Manage Expectation Suites
An Expectation Suite contains a group of Expectations that describe the same set of data. All the Expectations that you apply to your data are grouped into an Expectation Suite.
Prerequisites
- An installation of Python, version 3.8 to 3.11.
- An installation of GX 1.0.
- Recommended. A preconfigured Data Context.
- Recommended. A preconfigured Data Source and Data Asset connected to your data.
Create an Expectation Suite
- Procedure
- Sample code
- Import the GX Core library and the
ExpectationSuite
class:
import great_expectations as gx
from great_expectations.core.expectation_suite import ExpectationSuite
In this example the variable context
is your Data Context.
- Create an Expectation Suite:
new_suite_name = "my_first_expectation_suite"
suite = ExpectationSuite(name=new_suite_name)
- Add the Expectation Suite to your Data Context:
suite = context.suites.add(suite)
You can add an Expectation Suite to your Data Context at the same time as you create the Expectation Suite with the following code:
new_suite_name = "my_second_expectation_suite"
suite = context.suites.add(ExpectationSuite(name=new_suite_name))
import great_expectations as gx
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
new_suite_name = "my_first_expectation_suite"
suite = ExpectationSuite(name=new_suite_name)
suite = context.suites.add(suite)
new_suite_name = "my_second_expectation_suite"
suite = context.suites.add(ExpectationSuite(name=new_suite_name))
Get an existing Expectation Suite
- Procedure
- Sample code
In this example the variable context
is your Data Context.
- Use the Data Context to retrieve the existing Expectation Suite:
existing_suite_name = (
"my_expectation_suite" # replace this with the name of your Expectation Suite
)
suite = context.suites.get(name=existing_suite_name)
import great_expectations as gx
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
new_suite_name = "my_expectation_suite"
context.suites.add(ExpectationSuite(name=new_suite_name))
existing_suite_name = (
"my_expectation_suite" # replace this with the name of your Expectation Suite
)
suite = context.suites.get(name=existing_suite_name)
Rename an Expectation Suite
- Procedure
- Sample code
- Get the Expectation Suite to rename. This could be an existing Expectation Suite you retrieve from your Data Context or a new Expectation Suite that you have referenced earlier in your code.
In this example the variable suite
is your Expectation Suite.
- Change the
name
attribute of the Expectation Suite:
suite.name = "my_renamed_expectation_suite"
- Save the changes to the Expectation Suite:
suite.save()
The suite.save()
method will save all changes to the Expectation Suite, including changes that you have made to any Expectations within the Expectation Suite. If you have unsaved changes to Expectations in the Expectation Suite that you do not wish to keep, you should rename the Expectation Suite in a new Python session.
import great_expectations as gx
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
suite_name = "my_expectation_suite"
suite = context.suites.add(ExpectationSuite(name=suite_name))
suite.name = "my_renamed_expectation_suite"
suite.save()
Delete an Expectation Suite
- Procedure
- Sample code
In this example the variable context
is your Data Context.
- Get the Expectation Suite to delete:
suite_name = "my_deletable_expectation_suite"
suite = context.suites.get(suite_name)
- Use the Data Context to delete the retrieved Expectation Suite:
context.suites.delete(suite=suite)
import great_expectations as gx
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
new_suite_name = "my_deletable_expectation_suite"
new_suite = ExpectationSuite(name=new_suite_name)
context.suites.add(new_suite)
suite_name = "my_deletable_expectation_suite"
suite = context.suites.get(suite_name)
context.suites.delete(suite=suite)
Add Expectations to an Expectation Suite
- Procedure
- Sample code
- Create a new or get an existing Expectation Suite.
In this example the variable suite
is your Expectation Suite.
In this example the variable expectation
is the Expectation to add to the Expectation Suite.
- Add the Expectation to the Expectation Suite:
suite.add_expectation(expectation)
You can create an Expectation at the same time as you add it to the Expectation Suite:
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="pickup_datetime"))
import great_expectations as gx
import great_expectations.expectations as gxe
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
suite = context.suites.add(ExpectationSuite(name="my_expectation_suite"))
expectation = gxe.ExpectColumnValuesToBeInSet(
column="passenger_count", value_set=[1, 2, 3, 4, 5]
)
suite.add_expectation(expectation)
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="pickup_datetime"))
Get an Expectation from an Expectation Suite
- Procedure
- Sample code
- Get an existing Expectation Suite that contains Expectations or add some Expectations to a new Expectation Suite.
In this example the variable suite
is your Expectation Suite.
- Find the desired Expectation by iterating through the Expectations in the Expectation Suite and comparing classes and attributes to those of the desired Expectation:
expectation = next(
expectation
for expectation in suite.expectations
if isinstance(expectation, gxe.ExpectColumnValuesToNotBeNull)
and expectation.column == "pickup_datetime"
)
import great_expectations as gx
import great_expectations.expectations as gxe
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
suite = context.suites.add(ExpectationSuite(name="my_expectation_suite"))
suite.add_expectation(
gxe.ExpectColumnValuesToBeInSet(column="passenger_count", value_set=[1, 2, 3, 4, 5])
)
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="pickup_datetime"))
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="passenger_count"))
expectation = next(
expectation
for expectation in suite.expectations
if isinstance(expectation, gxe.ExpectColumnValuesToNotBeNull)
and expectation.column == "pickup_datetime"
)
Edit a single Expectation in an Expectation Suite
- Procedure
- Sample code
- Get the Expectation to edit from its Expectation Suite.
In this example the variable expectation
is the Expectation you want to edit.
expectation.column = "pickup_location_id"
- Save the modified Expectation in the Expectation Suite:
expectation.save()
expectation.save()
is explicitly used to update the configuration of an Expectation in an Expectation Suite.
An Expectation Suite continues to use the Expectation's original values unless you save your modifications. You can test changes to an Expectation without running expectation.save()
, but those changes will not persist in the Expectation Suite until expectation.save()
is run.
import great_expectations as gx
import great_expectations.expectations as gxe
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
new_suite_name = "my_expectation_suite"
new_suite = ExpectationSuite(name=new_suite_name)
context.suites.add(new_suite)
new_suite.add_expectation(
gxe.ExpectColumnValuesToBeInSet(column="passenger_count", value_set=[1, 2, 3, 4, 5])
)
new_suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="pickup_datetime"))
new_suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="passenger_count"))
existing_suite_name = (
"my_expectation_suite" # replace this with the name of your Expectation Suite
)
suite = context.suites.get(name=existing_suite_name)
expectation = next(
expectation
for expectation in suite.expectations
if isinstance(expectation, gxe.ExpectColumnValuesToNotBeNull)
and expectation.column == "pickup_datetime"
)
expectation.column = "pickup_location_id"
expectation.save()
Edit multiple Expectations in an Expectation Suite
- Procedure
- Sample code
- Get an existing Expectation Suite that contains Expectations, or add some Expectations to a new Expectation Suite.
In this example the variable suite
is your Expectation Suite.
- Modify multiple Expectations in the Expectation Suite:
for expectation in suite.expectations:
expectation.notes = "This Expectation was generated as part of GX Documentation."
- Save the Expectation Suite and all modifications to the Expectations within it:
suite.save()
import great_expectations as gx
import great_expectations.expectations as gxe
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
suite = context.suites.add(suite=ExpectationSuite(name="my_expectation_suite"))
suite.add_expectation(
gxe.ExpectColumnValuesToBeInSet(column="passenger_count", value_set=[1, 2, 3, 4, 5])
)
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="pickup_datetime"))
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="passenger_count"))
for expectation in suite.expectations:
expectation.notes = "This Expectation was generated as part of GX Documentation."
suite.save()
Delete an Expectation from an Expectation Suite
- Procedure
- Sample code
In this example the variable suite
is the Expectation Suite containing the Expectation to delete.
In this example the variable expectation
is the Expectation to delete.
- Use the Expectation Suite to delete the Expectation:
suite.delete_expectation(expectation=expectation_to_delete)
import great_expectations as gx
import great_expectations.expectations as gxe
from great_expectations.core.expectation_suite import ExpectationSuite
context = gx.get_context()
suite = ExpectationSuite(name="my_expectation_suite")
suite.add_expectation(
gxe.ExpectColumnValuesToBeInSet(column="passenger_count", value_set=[1, 2, 3, 4, 5])
)
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="pickup_datetime"))
suite.add_expectation(gxe.ExpectColumnValuesToNotBeNull(column="passenger_count"))
context.suites.add(suite)
expectation_to_delete = next(
expectation
for expectation in suite.expectations
if isinstance(expectation, gxe.ExpectColumnValuesToNotBeNull)
and expectation.column == "pickup_datetime"
)
suite.delete_expectation(expectation=expectation_to_delete)