# SpatioTemporal Asset Catalog Validator

[![PyPI Downloads](https://static.pepy.tech/personalized-badge/stac-validator?period=total&units=NONE&left_color=GREY&right_color=BLUE&left_text=downloads)](https://pepy.tech/projects/stac-validator)
  [![GitHub contributors](https://img.shields.io/github/contributors/stac-utils/stac-validator?color=blue)](https://github.com/stac-utils/stac-validator/graphs/contributors)
  [![GitHub stars](https://img.shields.io/github/stars/stac-utils/stac-validator.svg?color=blue)](https://github.com/stac-utils/stac-validator/stargazers)
  [![GitHub forks](https://img.shields.io/github/forks/stac-utils/stac-validator.svg?color=blue)](https://github.com/stac-utils/stac-validator/network/members)
   [![PyPI version](https://img.shields.io/pypi/v/stac-valid.svg?color=blue)](https://pypi.org/project/stac-valid/)
  [![STAC](https://img.shields.io/badge/STAC-1.1.0-blue.svg)](https://github.com/radiantearth/stac-spec/tree/v1.1.0)

> ⚠️ **IMPORTANT NOTICE: PyPI Package Renamed** ⚠️
>
> Due to an administrative transition, the PyPI package name for this project has changed. The `stac-validator` package on PyPI is no longer maintained. 
>
> To get the latest updates, bug fixes, and features (v4.2.0 and beyond), you must now install **`stac-valid`**.
>
> ### 🛠️ What you need to do
> 
> You only need to update your installation command or `requirements.txt`:
> ```bash
> # OLD
> pip install stac-validator
> 
> # NEW
> pip install stac-valid
> ```
> 
> **Your existing Python code and CLI scripts DO NOT need to change.** > You will still use `import stac_validator` and the `stac-validator` CLI command exactly as you did before.

## Table of Contents

- [Overview](#overview)
- [Documentation](#documentation)
- [Requirements](#requirements)
- [Installation](#install)
- [Supported STAC Versions](#versions-supported)
- [Usage](#usage)
  - [CLI](#cli)
    - [Legacy Validation](#legacy-validation)
  - [Batch Validation](#batch-validation)
  - [Fast Validation](#fast-validation)
  - [API Server (FastAPI)](#api-server-fastapi)
  - [Python](#python)
- [Schema Cache Settings](#schema-cache-settings)
- [Performance Benchmarking](#performance-benchmarking)
- [Examples](#additional-examples)
  - [Core Validation](#--core)
  - [Custom Schema](#--custom)
  - [Extensions Validation](#--extensions)
  - [Recursive Validation](#--recursive)
  - [Item Collection Validation](#--item-collection)
  - [Using Headers](#--header)
  - [Schema Mapping](#--schema-map)
  - [Schema Config](#--schema-config)
  - [Pydantic Validation](#--pydantic)
- [Deployment](#deployment)
  - [Docker](#docker)
  - [AWS (CDK)](#aws-cdk)
- [Testing](#testing)
- [Related Projects](#related-projects)
- [Sponsors and Supporters](#sponsors-and-supporters)
- [Contributing](#contributing)
- [License](#license)

## Overview

STAC Validator is a tool to validate [STAC (SpatioTemporal Asset Catalog)](https://github.com/radiantearth/stac-spec) json files against the official STAC specification. It provides both a command-line interface and a Python API for validating STAC objects.

## Documentation

For detailed documentation, please visit our [GitHub Pages documentation site](https://stac-utils.github.io/stac-validator/).

## Validate STAC json files against the [STAC spec](https://github.com/radiantearth/stac-spec).

```bash
$ stac-validator https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json
```

```bash
[
    {
        "version": "1.0.0",
        "path": "https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json",
        "schema": [
            "https://stac-extensions.github.io/eo/v1.0.0/schema.json",
            "https://stac-extensions.github.io/projection/v1.0.0/schema.json",
            "https://stac-extensions.github.io/scientific/v1.0.0/schema.json",
            "https://stac-extensions.github.io/view/v1.0.0/schema.json",
            "https://stac-extensions.github.io/remote-data/v1.0.0/schema.json",
            "https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "default"
    }
]
```

## Requirements

- Python 3.8+
  - Requests
  - Click
  - Jsonschema

## Related Projects

Note: Stac-validator is also used in stac-check which adds linting messages based on the official STAC best practices document.  
https://github.com/stac-utils/stac-check

## Install

### Installation from PyPi

```bash
$ pip install stac-valid
```

### Installation from Repo

```bash
$ pip install .
```

or for local development

```bash
$ pip install -e '.[dev]'
```

The [Makefile](./Makefile) has convenience commands if Make is installed.

```bash
$ make help
```

## Versions supported

| STAC         |
| ------------ |
| 0.8.0        |
| 0.8.1        |
| 0.9.0        |
| 1.0.0-beta.1 |
| 1.0.0-beta.2 |
| 1.0.0-rc.1   |
| 1.0.0-rc.2   |
| 1.0.0-rc.3   |
| 1.0.0-rc.4   |
| 1.0.0        |
| 1.1.0-beta.1 |
| 1.1.0        |

## Usage

### CLI

**Basic Usage**

```bash
$ stac-validator --help
```

```bash
Usage: stac-validator [OPTIONS] COMMAND [ARGS]...

  STAC Validator - Validate STAC files against the STAC specification.

  Usage:
    stac-validator validate <file> [options]
    stac-validator batch <files> [options]
    stac-validator batch <file> --item-collection [options]
      

Options:
  --help  Show this message and exit.

Commands:
  batch     Validate multiple STAC files concurrently using all available...
  validate  Main function for the `stac-validator` command line tool.
```

**Validate Command**

```bash
$ stac-validator validate --help
```

```bash
Usage: stac-validator validate [OPTIONS] STAC_FILE

  Validate a STAC file against the STAC specification.

  Prints validation results to the console as JSON. Exits with status code 0
  if valid, 1 if invalid.

Options:
  --core                          Validate core stac object only without
                                  extensions.
  --extensions                    Validate extensions only.
  --links                         Additionally validate links. Only works with
                                  default mode.
  --assets                        Additionally validate assets. Only works
                                  with default mode.
  -c, --custom TEXT               Validate against a custom schema (local
                                  filepath or remote schema).
  -sc, --schema-config TEXT       Validate against a custom schema config
                                  (local filepath or remote schema config).
  -s, --schema-map <TEXT TEXT>...
                                  Schema path to replaced by (local) schema
                                  path during validation. Can be used multiple
                                  times.
  -r, --recursive                 Recursively validate all related stac
                                  objects.
  -m, --max-depth INTEGER         Maximum depth to traverse when recursing.
                                  Omit this argument to get full recursion.
                                  Ignored if `recursive == False`.
  --collections                   Validate /collections response.
  --item-collection               Validate item collection response. Can be
                                  combined with --pages. Defaults to one page.
  --no-assets-urls                Disables the opening of href links when
                                  validating assets (enabled by default).
  --header <TEXT TEXT>...         HTTP header to include in the requests. Can
                                  be used multiple times.
  -p, --pages INTEGER             Maximum number of pages to validate via
                                  --item-collection. Defaults to one page.
  -t, --trace-recursion           Enables verbose output for recursive mode.
  --no_output                     Do not print output to console.
  --log_file TEXT                 Save full recursive output to log file
                                  (local filepath).
  --pydantic                      Validate using stac-pydantic models for
                                  enhanced type checking and validation.
  --verbose                       Enable verbose output. This will output
                                  additional information during validation.
  --schema-cache-size INTEGER     Max number of schema entries to cache in
                                  memory. Use 0 to disable schema caching.
                                  Defaults to 16.
  --help                          Show this message and exit.
```

**Batch Command**

```bash
$ stac-validator batch --help
```

```bash
Usage: stac-validator batch [OPTIONS] FILES...

  Validate multiple STAC files concurrently using all available CPU cores.

  This command uses multiprocessing to validate STAC files in parallel,
  bypassing Python's Global Interpreter Lock (GIL) for maximum performance.
  Each CPU core gets its own schema cache, which is warmed up on the first
  file and reused for subsequent files.

  Examples:

      # Validate all JSON files in a directory
      $ stac-validator batch *.json

      # Validate specific files
      $ stac-validator batch file1.json file2.json file3.json

      # Validate a GeoJSON FeatureCollection (validates each feature individually)
      $ stac-validator batch --item-collection sample_data/sentinel-cogs_0_100.json

      # Use only 4 cores
      $ stac-validator batch *.json --cores 4

      # Disable progress bar
      $ stac-validator batch *.json --no-progress

Options:
  --cores INTEGER       Number of CPU cores to use for parallel validation.
                        Defaults to all available cores.
  --no-progress         Disable progress bar during validation.
  --no-output           Do not print output to console.
  --item-collection  Treat files as GeoJSON FeatureCollections and validate
                        each feature individually.
  --verbose             Show full JSON output for all items. By default, only
                        invalid items are shown.
  --schema-cache-size INTEGER  Max number of schema entries to cache
                             per worker process. Use 0 to disable
                             schema caching. Defaults to 16.
  --batch-size INTEGER         Batch size for chunked processing. Larger
                               batches use more memory but may be faster.
                               Defaults to 2000.
  --help                Show this message and exit.
```

**Fast Command**  

```bash
$ stac-valid fast --help
```

```bash
Usage: stac-valid fast [OPTIONS] STAC_FILE

  High-speed validation using fastjsonschema and local caching.

Options:
  -q, --quiet    Suppress individual item logs.
  -v, --verbose  Show full validation logs for all items. By default, only
                 invalid items are shown.
    -r, --recursive  Recursively validate all child catalogs, collections,
                                     and items.
    -a, --api        Validate a STAC API catalog recursively (follows data,
                                     child, item, and items links).
    --limit INTEGER RANGE  Limit number of STAC objects to validate.
                                                 [x>=1]
  --help         Show this message and exit.
```

#### Legacy Validation

The `validate` command is the main legacy validation tool with comprehensive options:

```bash
# Basic single file validation
$ stac-validator validate path/to/stac_file.json

# Validate with custom schema
$ stac-validator validate item.json --custom /path/to/schema.json

# Recursively validate all related STAC objects
$ stac-validator validate catalog.json --recursive --max-depth 5

# Validate collections endpoint response
$ stac-validator validate https://example.com/collections --collections

# Validate item collection response
$ stac-validator validate https://example.com/search --item-collection --pages 10

# Validate with extensions and links
$ stac-validator validate item.json --extensions --links --assets
```

**Options include:**
- `--core` - Validate core STAC only (skip extensions)
- `--extensions` - Validate extensions only
- `--links` - Validate link objects
- `--assets` - Validate asset objects
- `--recursive` - Recursively validate related STAC objects
- `--custom` - Validate against custom schema
- `--schema-map` - Replace schema URLs during validation
- `--collections` - Validate /collections endpoint response
- `--item-collection` - Validate item collection responses
- `--pydantic` - Use Pydantic models for validation
- `--schema-cache-size` - Configure schema cache size
- `--batch-size` - Configure batch size for chunked processing (batch command only)
- And more (see `stac-validator validate --help`)

#### Batch Validation

The `batch` command validates multiple STAC files concurrently using multiprocessing to bypass Python's Global Interpreter Lock (GIL). This enables **10-100x performance improvement** over single-threaded validation by utilizing all available CPU cores.

**Architecture:**

- **Multiprocessing:** Each CPU core runs an independent Python process
- **Per-worker schema cache:** Each worker maintains its own LRU cache of downloaded schemas (default 16 per worker)
- **Cache warmup:** First file on each worker downloads schemas, subsequent files use cached copies
- **Configurable cache:** Use `--schema-cache-size` to adjust cache size per worker (0 = disabled)
- **Linear scaling:** Performance scales linearly with available cores (e.g., 8 cores = ~8x faster)
- **Container-aware:** Automatically detects Docker/ECS CPU limits via `os.sched_getaffinity()`

**Basic Usage**

```bash
# Validate all JSON files in current directory
$ stac-validator batch *.json

# Validate specific files
$ stac-validator batch item1.json item2.json item3.json

# Validate a FeatureCollection (extracts and validates each feature)
$ stac-validator batch collection.json --item-collection
```

**Options**

```bash
# Use specific number of cores
$ stac-validator batch *.json --cores 4

# Reserve cores for OS (useful on local machines)
$ stac-validator batch *.json --cores -1  # Uses all cores minus 1

# Disable progress bar (for CI/CD environments)
$ stac-validator batch *.json --no-progress

# Suppress JSON output (show only summary)
$ stac-validator batch *.json --no-output

# Configure schema cache size per worker (default: 16 schemas per worker)
$ stac-validator batch *.json --schema-cache-size 32

# Disable schema caching entirely
$ stac-validator batch *.json --schema-cache-size 0

# Configure batch size for chunked processing (default: 2000 items per chunk)
$ stac-validator batch *.json --batch-size 5000

# Use larger batch size for faster processing (uses more memory)
$ stac-validator batch collection.json --item-collection --batch-size 10000

# Use smaller batch size for memory-constrained environments
$ stac-validator batch *.json --batch-size 500
```

**Batch Size Configuration**

The `--batch-size` option controls how many items are processed in each chunk. This affects memory usage and performance:

- **Default (2000):** Balanced memory usage and performance for most systems
- **Larger values (5000-10000):** Faster processing on systems with abundant memory; reduces overhead from creating multiple worker pools
- **Smaller values (500-1000):** Lower memory footprint; useful on memory-constrained systems or when validating very large items

**How It Works**

1. **Startup:** Detects available CPU cores (respects Docker limits)
2. **Distribution:** Distributes files across worker processes
3. **Validation:** Each worker validates files independently
4. **Schema Caching:** Each worker maintains its own LRU cache (default 16 schemas per worker)
   - First file on a worker: schemas are fetched and cached
   - Subsequent files: schemas are reused from cache (no network/disk I/O)
   - Total memory: up to `cores × schema_cache_size` schemas in memory
5. **Results:** Aggregates results and displays summary statistics

**Example Output**

```bash
$ stac-validator batch --item-collection sample_data/sentinel-cogs_0_100.json
[
    {
        "path": "sample_data/sentinel-cogs_0_100.json[0]",
        "valid_stac": false,
        "errors": [
            "'eo:bands' does not match any of the regexes: '^(?!eo:)'. Error is in properties "
        ]
    }
]

Validation Summary:
  Total files: 100
  CPU cores used: 16
  ✅ Valid: 99
  ❌ Invalid: 1

Failed validations:
  sample_data/sentinel-cogs_0_100.json[0]
    - 'eo:bands' does not match any of the regexes: '^(?!eo:)'. Error is in properties 

Validation completed in 1.10s
```

**Performance Characteristics**

| Scenario | Single-threaded | Batch (8 cores) | Speedup |
|----------|-----------------|-----------------|---------|
| 10000 items | ~130 seconds | ~22 seconds | 5.9x |

*Times vary based on schema complexity and network latency for first download*

**Python API**

```python
from stac_validator.batch_validator import validate_concurrently

# Validate files
results = validate_concurrently(
    ["item1.json", "item2.json", "item3.json"],
    max_workers=None,  # Auto-detect cores
    show_progress=True
)

# Validate FeatureCollections
results = validate_concurrently(
    ["collection.json"],
    feature_collection=True,
    max_workers=8
)

# Process results
for result in results:
    if result["valid_stac"]:
        print(f"✅ {result['path']}")
    else:
        print(f"❌ {result['path']}")
        for error in result.get("errors", []):
            print(f"   - {error}")
```

**Use Cases**

- **Bulk ingestion:** Validate thousands of STAC items from ESA, Copernicus, or other catalogs
- **CI/CD pipelines:** Validate entire dataset collections in GitHub Actions or AWS CodePipeline
- **Data quality checks:** Periodically validate all items in a STAC catalog
- **Migration validation:** Verify all items when upgrading STAC versions
- **API preprocessing:** Validate incoming FeatureCollections before storage (see FastAPI integration)

#### Fast Validation

The `fast` command provides ultra-high-speed validation using `fastjsonschema` with intelligent caching. It's optimized for large FeatureCollections, delivering **batch-like performance with significantly lower memory overhead** by using a single-threaded, compiled validator approach instead of multiprocessing.

**Key Features:**

- **fastjsonschema:** Compiled validators for 10-100x faster validation than standard jsonschema
- **Single-threaded:** No multiprocessing overhead - ideal for memory-constrained environments
- **Batch-comparable speed:** Similar throughput to batch command but with minimal memory footprint
- **Multi-tier caching:** RAM → Disk → Network with automatic fallback
- **Local schema storage:** Schemas cached locally under `local_schemas/.schemas` directory for instant reuse
- **Automatic detection:** Detects STAC type (Item, Collection, Catalog, FeatureCollection) automatically
- **Recursive traversal:** Supports `--recursive` for local catalog/collection graphs
- **STAC API traversal:** Supports `--api` to follow STAC API data, child, item, and items links
- **Detailed metrics:** Shows setup time, execution time, and cache hit status for each item
- **Error grouping:** Groups validation errors by type and shows affected items

**Basic Usage**

```bash
# Validate a single STAC Item
$ stac-validator fast item.json

# Validate a FeatureCollection (validates each feature)
$ stac-validator fast collection.json

# Validate a STAC Collection
$ stac-validator fast collection-metadata.json

# Validate a STAC Catalog
$ stac-validator fast catalog.json
```

**Options**

```bash
# Suppress output (only show summary)
$ stac-validator fast item.json --quiet

# Show detailed output for all items (default shows first 5)
$ stac-validator fast collection.json --verbose

# Validate only first 25 objects in a large FeatureCollection
$ stac-validator fast collection.json --limit 25

# Recursively validate a local catalog graph
$ stac-validator fast catalog.json --recursive

# Recursively validate a STAC API root endpoint
$ stac-validator fast https://api.example.com --api

# Combine options
$ stac-validator fast collection.json --verbose --limit 50
```

**Example Output**

```bash
$ stac-validator fast sample_data/sentinel-cogs_0_100.json

📂 Loading: sample_data/sentinel-cogs_0_100.json
📦 Detected FeatureCollection (100 Items)

[1] ID: S2B_1CDH_20191211_0_L2A | Type: Item | Cache 🐌 | Setup: 125.34ms | Exec:  2.45ms | ✅ VALID
[2] ID: S2B_1CDH_20191220_0_L2A | Type: Item | Cache ⚡ | Setup:   0.12ms | Exec:  1.89ms | ✅ VALID
[3] ID: S2B_1CDH_20191230_0_L2A | Type: Item | Cache ⚡ | Setup:   0.08ms | Exec:  2.12ms | ❌ INVALID
[4] ID: S2B_1CCV_20191029_0_L2A | Type: Item | Cache ⚡ | Setup:   0.09ms | Exec:  1.95ms | ✅ VALID
[5] ID: S2B_1CCV_20191128_0_L2A | Type: Item | Cache ⚡ | Setup:   0.07ms | Exec:  2.03ms | ✅ VALID
... silencing output for remaining items (validating at maximum speed) ...

=======================================================
📊 VALIDATION SUMMARY
=======================================================
Total Objects Processed : 100
Valid Objects           : 46
Invalid Objects         : 54
-------------------------------------------------------
Total Setup Time        : 144.35 ms
Total Execution Time    : 25.74 ms
Average Exec per Object : 0.257 ms
=======================================================
🚨 ERROR BREAKDOWN
=======================================================

❌ STAC Spec Violation: Missing {'rel': 'collection'} in links array.
   Affected Items: 54
   Examples:       S2B_1CCV_20190102_0_L2A, S2B_1CCV_20190122_0_L2A, S2B_1CCV_20191029_0_L2A ... (and 51 more)
```

**Cache Indicators**

- **🐌 (Slow):** First validation - schemas being fetched and compiled
- **⚡ (Lightning):** Cached validator - instant execution using pre-compiled validator

**Schema Caching & Performance Improvement**

The fast validator uses a multi-tier caching strategy that dramatically speeds up subsequent runs:

1. **First Run:** Schemas are downloaded from the network and compiled with `fastjsonschema`
   - Setup time: 5-6 seconds (includes network fetch for all schemas + compilation)
   - Per-item execution: 0.25ms
   - Total for 100 items: ~5.4 seconds
   - Total for 1000 items: ~10-11 seconds

2. **Subsequent Runs:** Schemas are loaded from local disk cache (`.schemas/` directory)
   - Setup time: 100-200ms (instant disk read + compilation from cache)
   - Per-item execution: 0.25ms
   - Total for 100 items: ~150-200ms
   - Total for 1000 items: ~400-500ms (0.4-0.5 seconds)
   - **Speedup: 20-25x faster** compared to first run

**Cache Storage:**

Schemas are automatically cached in `local_schemas/.schemas/` directory:
```
local_schemas/
└── .schemas/
```

This means:
- ✅ No network calls on subsequent validations
- ✅ Instant schema availability across multiple runs
- ✅ Persistent cache survives process restarts
- ✅ Minimal disk space usage (schemas are typically <100KB each)

**Performance Characteristics**

The fast validator is optimized for:
- Large FeatureCollections (100-100,000+ items)
- Memory-constrained environments (single-threaded, no multiprocessing)
- Rapid iteration during development
- CI/CD pipelines where speed and memory efficiency are critical

Typical performance (subsequent runs with disk cache):
- **Setup time:** 100-200ms (disk read + compilation from cached schemas)
- **Per-item execution:** 0.23-0.25ms (compiled fastjsonschema validator)
- **Total time for 100 items:** ~150-200ms
- **Total time for 1000 items:** ~400-500ms (0.4-0.5 seconds)
- **Memory footprint:** Single process, minimal overhead (vs. batch's multiprocessing overhead)

**Batch vs. Fast Comparison**

| Metric | Batch | Fast |
|--------|-------|------|
| Memory | High (multiprocessing) | Low (single-threaded) |
| CPU cores | All available | Single core |
| Best for | Large systems with many cores | Memory-constrained or single-core systems |
| FeatureCollections | Excellent | Excellent |
| Large datasets | Excellent | Good |

**Python API**

```python
from stac_validator.fast_validator import FastValidator

# Validate a file
validator = FastValidator("item.json", quiet=False, verbose=False)
validator.run()

# Check if valid
if validator.valid:
    print("✅ Valid STAC")
else:
    print("❌ Invalid STAC")
```

**Choosing the Right Command**

| Use Case | Command | Why |
|----------|---------|-----|
| Single file validation | `validate` | Full feature set, detailed error messages, extensions support |
| Large FeatureCollections (memory-constrained) | `fast` | Batch-like speed, single-threaded, minimal memory overhead |
| Bulk validation (1000+ files, multi-core systems) | `batch` | Multiprocessing, utilizes all CPU cores, scales linearly |
| Quick validation with detailed metrics | `fast` | Ultra-fast with fastjsonschema, timing info per item, auto-detection |
| CI/CD pipelines (memory-limited) | `fast` | Fast, low memory, suitable for containers and serverless |
| CI/CD pipelines (resource-rich) | `batch` | Parallelization, maximum throughput, consistent output |
| Development/testing | `fast` | Instant feedback, detailed metrics, minimal overhead |
| Complex validation rules | `validate` | Full control over validation options, recursive validation |

#### API Server (FastAPI)

The `fast` validation engine can be deployed as a high-performance REST API, ideal for validating STAC objects during ingestion or as part of a microservices architecture.

**Running the Server (Local):**
```bash
# Requires fastapi and uvicorn
pip install "stac-valid[server]" 
python server/server.py
```

**Running the Server (Docker):**
```bash
# Pull and run the official GitHub container image
docker run -p 8000:8000 ghcr.io/staclabs/stac-validator:latest
```

**Validate via local script:**
```bash
python server/api_client_example.py sample_data/sentinel-cogs_0_100.json
```

**Validate via curl:**
```bash
curl -X POST http://localhost:8000/validate \
  -H "Content-Type: application/json" \
  -d @sample_data/sentinel-cogs_0_100.json
```

**Response Format:**
The API returns a detailed JSON summary including performance metrics and error breakdowns:
```json
{
  "path": "request_body",
  "valid_stac": true,
  "total_objects": 100,
  "valid_objects": 100,
  "invalid_objects": 0,
  "setup_time_ms": 0.25,
  "execution_time_ms": 25.4,
  "errors": []
}
```

### Python

**Single File Validation**

```python
from stac_validator import stac_validator

# Remote source
stac = stac_validator.StacValidate("https://raw.githubusercontent.com/stac-utils/pystac/main/tests/data-files/examples/0.9.0/collection-spec/examples/landsat-collection.json")
stac.run()
print(stac.message)

# Local file
stac = stac_validator.StacValidate("tests/test_data/1beta1/sentinel2.json", extensions=True)
stac.run()
print(stac.message)
```

**Dictionary Validation**

```python
from stac_validator import stac_validator

stac = stac_validator.StacValidate()
stac.validate_dict(item_dict)
print(stac.message)
```

**Batch Validation - List of Dictionaries**

For validating dictionaries directly without managing temp files, use `validate_dicts()`:

```python
from stac_validator.batch_validator import validate_dicts

items = [
    {"type": "Feature", "stac_version": "1.1.0", ...},
    {"type": "Feature", "stac_version": "1.1.0", ...},
    {"type": "Feature", "stac_version": "1.1.0", ...},
]

# Validate all items concurrently (temp files handled internally)
results = validate_dicts(items, max_workers=None, show_progress=True)

print(f"Total: {len(results)}")
print(f"Valid: {sum(1 for r in results if r['valid_stac'])}")
print(f"Invalid: {sum(1 for r in results if not r['valid_stac'])}")

# Process results
for result in results:
    if result["valid_stac"]:
        print(f"✅ Valid")
    else:
        print(f"❌ Invalid: {result.get('errors', [])}")
```

**Parameters:**
- `items` - List of STAC item dictionaries
- `max_workers` - CPU cores to use (None = auto-detect, positive int = specific cores, negative int = all minus N)
- `show_progress` - Display progress bar (default: True)
- `chunk_size` - Number of items to process at a time to bound disk and memory usage (default: 1000)

**Batch Validation - FeatureCollection (Concurrent with Multiprocessing)**

For FeatureCollection validation with multiprocessing, use `validate_concurrently()` with `feature_collection=True`:

```python
from stac_validator.batch_validator import validate_concurrently

# Validate FeatureCollection files directly (10-100x faster for large collections)
results = validate_concurrently(
    ["collection1.json", "collection2.json"],
    feature_collection=True,  # Expand and validate each feature
    max_workers=None  # Auto-detect cores
)

print(f"Total features: {len(results)}")
print(f"Valid: {sum(1 for r in results if r['valid_stac'])}")
print(f"Invalid: {sum(1 for r in results if not r['valid_stac'])}")

# Process results
for result in results:
    if result["valid_stac"]:
        print(f"✅ Feature valid")
    else:
        print(f"❌ Feature invalid: {result.get('errors', [])}")
```

**Batch Validation - Multiple Files**

```python
from stac_validator.batch_validator import validate_concurrently

files = [
    "item1.json",
    "item2.json",
    "item3.json",
]

# Validate files concurrently using all available CPU cores
results = validate_concurrently(
    files,
    max_workers=None,  # Auto-detect cores
    show_progress=True
)

# Process results
for result in results:
    if result["valid_stac"]:
        print(f"✅ {result['path']}")
    else:
        print(f"❌ {result['path']}: {result.get('errors', [])}")
```

**Batch Validation - FeatureCollection Files**

```python
from stac_validator.batch_validator import validate_concurrently

files = ["collection1.json", "collection2.json"]

# Validate FeatureCollections by extracting and validating each feature
results = validate_concurrently(
    files,
    feature_collection=True,
    max_workers=8
)

# Results show feature index: "collection1.json[0]", "collection1.json[1]", etc.
for result in results:
    print(f"{result['path']}: {'✅' if result['valid_stac'] else '❌'}")
```

**Item Collection Validation**

```python
from stac_validator import stac_validator

stac = stac_validator.StacValidate()
stac.validate_item_collection_dict(item_collection_dict)
print(stac.message)
```

**Fast Validation (High-Speed with Local Caching)**

For large FeatureCollections or when you need maximum speed with minimal memory overhead:

```python
from stac_validator.fast_validator import FastValidator
import json

# Validate a FeatureCollection or single STAC object
fv = FastValidator("large_collection.json", quiet=True)
fv.run()

# Optionally cap validation to the first N objects
fv_limited = FastValidator("large_collection.json", quiet=True, limit=100)
fv_limited.run()

# Access validation results via the message attribute
print(json.dumps(fv.message, indent=2))

# Example output:
# {
#   "path": "large_collection.json",
#   "valid_stac": true,
#   "stac_versions": ["1.0.0", "1.1.0"],
#   "schemas_checked": [
#     "https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json",
#     "https://schemas.stacspec.org/v1.1.0/item-spec/json-schema/item.json",
#     "https://stac-extensions.github.io/projection/v1.0.0/schema.json"
#   ],
#   "total_objects": 1000,
#   "valid_objects": 998,
#   "invalid_objects": 2,
#   "setup_time_ms": 145.32,
#   "execution_time_ms": 245.67,
#   "errors": [
#     {
#       "error_message": "STAC Spec Violation: Missing {'rel': 'collection'} in links array.",
#       "affected_items": ["item_1", "item_2"],
#       "count": 2
#     }
#   ]
# }

# Check validity
if fv.valid:
    print(f"✅ All {fv.message[0]['valid_objects']} items are valid")
else:
    print(f"❌ {fv.message[0]['invalid_objects']} items failed validation")
    for error in fv.message[0]['errors']:
        print(f"  - {error['error_message']}: {error['count']} items affected")
```

**Configure Schema Cache Size**

```python
from stac_validator import stac_validator
from stac_validator.utilities import set_schema_cache_size

# Set once at app startup (process-wide)
set_schema_cache_size(16)  # use 0 to disable caching

stac = stac_validator.StacValidate()
stac.validate_dict(dictionary)
print(stac.message)
```


### Schema Cache Settings

- Default schema cache size is 16 entries.
- Use `--schema-cache-size` in the CLI or `set_schema_cache_size(...)` in Python to override it.
- Use `0` to disable schema caching.

Use `set_schema_cache_size` once at application startup:

```python
from stac_validator.utilities import set_schema_cache_size

# Examples:
set_schema_cache_size(16)  # small cache for low-memory deployments
set_schema_cache_size(64)  # moderate cache for long-running services
set_schema_cache_size(0)   # disable schema caching
```

Notes:
- `StacValidate()` and `validate_dict()` do not accept a cache-size parameter.
- Changing cache size at runtime replaces the cache instance and drops existing cached entries.
- In multi-worker deployments, configure cache size in each worker process.

## Performance Benchmarking

A benchmark script is included to compare the performance of batch validation vs legacy item-collection validation. This is useful for understanding the performance improvements of the multiprocessing batch validator.

### Running the Benchmark

```bash
# Test with 10,000 items (default)
python benchmark_validation.py

# Test with custom number of items
python benchmark_validation.py --items 5000
python benchmark_validation.py --items 50000

# Run only batch validation (skip slow legacy validation)
python benchmark_validation.py --items 10000 --batch-only

# Run only legacy validation
python benchmark_validation.py --items 10000 --legacy-only

# View all options
python benchmark_validation.py --help
```

### Example Output

```
======================================================================
BENCHMARK RESULTS
======================================================================
Items tested: 10000

Batch Validation (multiprocessing):
  Time: 22.43s
  ✅ Valid: 0
  ❌ Invalid: 10000

Legacy Validation (single-threaded):
  Time: 131.62s
  ✅ Valid: 0
  ❌ Invalid: 10000

======================================================================
Speedup: 5.9x faster with batch validation
Time saved: 109.19s
```

### Interpreting Results

- **Speedup Factor**: Shows how many times faster batch validation is compared to legacy validation
- **Time Saved**: The absolute time difference in seconds
- **CPU Cores Used**: Number of CPU cores utilized by batch validation (typically all available cores)

The batch validator's multiprocessing approach provides significant performance improvements, especially for large datasets. The speedup factor varies based on:
- Number of CPU cores available
- Size of the dataset
- Complexity of validation (extensions, custom schemas, etc.)

## Deployment

### Docker

The validator can run using docker containers.

```bash
$ docker build -t stac-validator .
$ docker run stac-validator https://raw.githubusercontent.com/stac-extensions/projection/main/examples/item.json
```

```bash
[
    {
        "version": "1.0.0",
        "path": "https://raw.githubusercontent.com/stac-extensions/projection/main/examples/item.json",
        "schema": [
            "https://stac-extensions.github.io/projection/v1.0.0/schema.json",
            "https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "default"
    }
]
```

### AWS (CDK)

An example [AWS CDK](https://aws.amazon.com/cdk/) deployment is available in [cdk-deployment](./cdk-deployment/README.md)

```bash
$ cd cdk-deployment
$ cdk diff
```

## Testing

```bash
$ make test
# or
$ pytest -v
```

See the [tests](./tests/test_stac_validator.py) files for examples on different usages.

## Additional Examples

### --core

```bash
$ stac-validator https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json --core
```

```bash
[
    {
        "version": "1.0.0",
        "path": "https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json",
        "schema": [
            "https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "core"
    }
]
```

### --custom

```bash
$ stac-validator https://radarstac.s3.amazonaws.com/stac/catalog.json --custom https://cdn.staclint.com/v0.7.0/catalog.json
```

```bash
[
    {
        "version": "0.7.0",
        "path": "https://radarstac.s3.amazonaws.com/stac/catalog.json",
        "schema": [
            "https://cdn.staclint.com/v0.7.0/catalog.json"
        ],
        "asset_type": "CATALOG",
        "validation_method": "custom",
        "valid_stac": true
    }
]
```

### --extensions

```bash
$ stac-validator https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json --extensions
```

```bash
[
    {
        "version": "1.0.0",
        "path": "https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json",
        "schema": [
            "https://stac-extensions.github.io/eo/v1.0.0/schema.json",
            "https://stac-extensions.github.io/projection/v1.0.0/schema.json",
            "https://stac-extensions.github.io/scientific/v1.0.0/schema.json",
            "https://stac-extensions.github.io/view/v1.0.0/schema.json",
            "https://stac-extensions.github.io/remote-data/v1.0.0/schema.json"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "extensions"
    }
]
```

### --recursive

```bash
$ stac-validator https://spot-canada-ortho.s3.amazonaws.com/catalog.json --recursive --max-depth 1 --trace-recursion
```

```bash
[
    {
        "version": "0.8.1",
        "path": "https://canada-spot-ortho.s3.amazonaws.com/canada_spot_orthoimages/canada_spot4_orthoimages/collection.json",
        "schema": "https://cdn.staclint.com/v0.8.1/collection.json",
        "asset_type": "COLLECTION",
        "validation_method": "recursive",
        "valid_stac": true
    },
    {
        "version": "0.8.1",
        "path": "https://canada-spot-ortho.s3.amazonaws.com/canada_spot_orthoimages/canada_spot5_orthoimages/collection.json",
        "schema": "https://cdn.staclint.com/v0.8.1/collection.json",
        "asset_type": "COLLECTION",
        "validation_method": "recursive",
        "valid_stac": true
    },
    {
        "version": "0.8.1",
        "path": "https://spot-canada-ortho.s3.amazonaws.com/catalog.json",
        "schema": "https://cdn.staclint.com/v0.8.1/catalog.json",
        "asset_type": "CATALOG",
        "validation_method": "recursive",
        "valid_stac": true
    }
]
```

### --item-collection

```bash
$ stac-validator https://earth-search.aws.element84.com/v0/collections/sentinel-s2-l2a/items --item-collection --pages 2
```

### --header

```bash
$ stac-validator https://stac-catalog.eu/collections/sentinel-s2-l2a/items --header x-api-key $MY_API_KEY --header foo bar
```

### --schema-map

Schema map allows stac-validator to replace a schema in a STAC json by a schema from another URL or local schema file.
This is especially useful when developing a schema and testing validation against your local copy of the schema.

```bash
$ stac-validator https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0/examples/extended-item.json --extensions --schema-map https://stac-extensions.github.io/projection/v1.0.0/schema.json "tests/test_data/schema/v1.0.0/projection.json"
```

```bash
[
    {
        "version": "1.0.0",
        "path": "https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0/examples/extended-item.json",
        "schema": [
            "https://stac-extensions.github.io/eo/v1.0.0/schema.json",
            "tests/test_data/schema/v1.0.0/projection.json",
            "https://stac-extensions.github.io/scientific/v1.0.0/schema.json",
            "https://stac-extensions.github.io/view/v1.0.0/schema.json",
            "https://stac-extensions.github.io/remote-data/v1.0.0/schema.json"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "extensions"
    }
]
```

This option is also capable of replacing URLs to subschemas:

```bash
$ stac-validator tests/test_data/v100/extended-item-local.json --custom tests/test_data/schema/v1.0.0/item_with_unreachable_url.json --schema-map https://geojson-wrong-url.org/schema/Feature.json https://geojson.org/schema/Feature.json --schema-map https://geojson-wrong-url.org/schema/Geometry.json https://geojson.org/schema/Geometry.json
```

```bash
[
    {
        "version": "1.0.0",
        "path": "tests/test_data/v100/extended-item-local.json",
        "schema": [
            "tests/test_data/schema/v1.0.0/item_with_unreachable_url.json"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "custom"
    }
]
```

### --schema-config

The `--schema-config` option allows you to specify a YAML or JSON configuration file that maps remote schema URLs to local file paths. This is useful when you need to validate against multiple local schemas and want to avoid using multiple `--schema-map` arguments.

Example schema config file (YAML):
```yaml
schemas:
  "https://schemas.stacspec.org/v1.0.0/collection-spec/json-schema/collection.json": "local_schemas/v1.0.0/collection.json"
  "https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/item.json": "local_schemas/v1.0.0/item.json"
  "https://stac-extensions.github.io/eo/v1.0.0/schema.json": "local_schemas/v1.0.0/eo.json"
```

Usage:
```bash
$ stac-validator https://raw.githubusercontent.com/radiantearth/stac-spec/v1.0.0/examples/extended-item.json --schema-config path/to/schema_config.yaml
```

The paths in the config file can be absolute or relative to the config file's location.

### --pydantic

The `--pydantic` option provides enhanced validation using stac-pydantic models, which offer stronger type checking and more detailed error messages. To use this feature, you need to install the optional dependency:

```bash
$ pip install stac-valid[pydantic]
```

Then you can validate your STAC objects using Pydantic models:

```bash
$ stac-validator https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json --pydantic
```

```bash
[
    {
        "version": "1.0.0",
        "path": "https://raw.githubusercontent.com/radiantearth/stac-spec/master/examples/extended-item.json",
        "schema": [
            "stac-pydantic Item model"
        ],
        "valid_stac": true,
        "asset_type": "ITEM",
        "validation_method": "pydantic",
        "extension_schemas": [
            "https://stac-extensions.github.io/eo/v1.0.0/schema.json",
            "https://stac-extensions.github.io/projection/v1.0.0/schema.json",
            "https://stac-extensions.github.io/scientific/v1.0.0/schema.json",
            "https://stac-extensions.github.io/view/v1.0.0/schema.json",
            "https://stac-extensions.github.io/remote-data/v1.0.0/schema.json"
        ],
        "model_validation": "passed"
    }
]
```

## Sponsors and Supporters

The following organizations have contributed time and/or funding to support the development of this project:
- [Healy Hyperspatial](https://healy-hyperspatial.github.io/)
- [Radiant Earth Foundation](https://radiant.earth/)
- [Sparkgeo](https://sparkgeo.com/)
- [CloudFerro](https://cloudferro.com/)

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

This project is licensed under the Apache License 2.0.
