Saving and sharing results¶

This page explains how to save your analysis outputs to the workspace catalog so they persist between sessions and can be shared with colleagues in the same workspace.

Why save to the workspace catalog?¶

Results persist between sessions
Other workspace members can query your output tables
Keeps derived data separate from the read-only source mart

You cannot write to the mart catalog

<mart_catalog_name> is read-only. All outputs must be saved to <workspace_catalog_name>.

1. Create a project schema¶

Create a schema to hold your project's tables. Run this once.

WORKSPACE_CATALOG = "<workspace_catalog_name>"
PROJECT_SCHEMA = "my_project"  # edit this

spark.sql(f"CREATE SCHEMA IF NOT EXISTS {WORKSPACE_CATALOG}.{PROJECT_SCHEMA}")
print(f"Schema ready: {WORKSPACE_CATALOG}.{PROJECT_SCHEMA}")

2. Save a Spark DataFrame as a table¶

# result = your DataFrame from earlier analysis

result.write.mode("overwrite").saveAsTable(
    f"{WORKSPACE_CATALOG}.{PROJECT_SCHEMA}.my_filtered_cohort"
)

Write modes¶

Mode	Behaviour
`overwrite`	Replaces the table entirely — use for reproducible outputs
`append`	Adds rows to an existing table — use for incremental loads
`ignore`	Does nothing if the table already exists
`error` (default)	Fails if the table already exists

3. Save a pandas DataFrame as a table¶

If you have been working in pandas, convert back to Spark first:

import pandas as pd
from pyspark.sql import SparkSession

# pdf is your pandas DataFrame
spark_df = spark.createDataFrame(pdf)

spark_df.write.mode("overwrite").saveAsTable(
    f"{WORKSPACE_CATALOG}.{PROJECT_SCHEMA}.my_pandas_result"
)

4. Save using SQL¶

-- Create a table directly from a query
CREATE OR REPLACE TABLE <workspace_catalog_name>.my_project.my_result AS
SELECT *
FROM <mart_catalog_name>.<schema_name>.<table_name>
WHERE column_c IS NOT NULL;

5. Verify your saved data¶

# Read it back to confirm
saved = spark.table(f"{WORKSPACE_CATALOG}.{PROJECT_SCHEMA}.my_filtered_cohort")
print(f"Saved rows: {saved.count():,}")
saved.show(5)

SELECT COUNT(*) FROM <workspace_catalog_name>.my_project.my_filtered_cohort;

Any table saved to <workspace_catalog_name> is accessible to all members of your workspace — they can query it directly in their own notebooks using the full table path <workspace_catalog_name>.<schema>.<table>.