Skip to main content
Facts in Oso Cloud need to stay in sync with your application data. This page covers:
  • Running an initial sync from your DB to Oso Cloud.
  • Keeping facts up-to-date in production with Oso Sync.
  • Configuration for PostgreSQL, MongoDB, and CSV.
  • Limitations and best practices.
For adding facts in development using the Fact Schema UI, or inserting facts manually during runtime, see Insert facts).

​
Sync facts in production

Oso Sync (available on Startup and Growth plans) updates Oso Cloud to match your application database. Typical flow:
  1. Decide how to represent authorization data as facts.
  2. Run a one-time initial sync to populate Oso Cloud.
  3. Keep facts in sync using dual writes and periodic reconciliation.

​
Initial sync with Oso Sync

Oso Sync connects to your data source(s) and runs a configured query for each fact type. Run from the CLI: 
oso-cloud reconcile --perform-updates reconcile.yaml

​
Configure Oso Sync

Your reconcile.yaml maps data sources to fact types. We currently support the following data sources: PostgreSQL, MongoDB, and Comma-separated Values (CSV).

​
PostgreSQL

version: 1
source: postgres
facts:
  has_relation(Repository:_, String:parent, Organization:_):
    db: app_db
    query: |-
      select repository.public_id, organization.public_id
      from repository
      join organization
        on organization.id = repository.organization_id
dbs:
  app_db:
    connection_string: postgresql://oso:oso@host:5432/dbname
Configuration details:
  • version: optional, defaults to 1.
  • source: optional, defaults to postgres.
  • facts:
    • Fact type uses positional variables (_) that map to query columns in order.
    • db matches an entry in dbs.
    • query returns all facts of that type.
    • Example: has_relation(Repository:_, String:parent, Organization:_) has variables in the first and third arguments. repository.public_id fills the first argument (Repository), and organization.public_id fills the third argument (Organization).
  • dbs:
    • Maps unique names to database connection details.
    • connection_string is a PostgreSQL connection URI or an environment variable: connection_string: $ENV_VAR_NAME.

​
MongoDB

version: 1
source: mongodb
facts:
  has_relation(Repository:_, String:parent, Organization:_):
    db: app_db
    collection: has_relation
    fields:
      - name: repository
      - name: organization
        is_array: true
    query:
      find: {}
dbs:
  app_db:
    connection_string: mongodb://oso:oso@somemongo.instance.aws.com:27017/dbname
Configuration details:
  • version: must be 1.
  • source: must be mongodb.
  • facts:
    • Fact type uses positional variables (_) that map to query columns in order.
    • collection: name containing fact data.
    • fields: maps to positional arguments in fact type.
      • At most one field can have is_array: true (automatically unwound).
    • query can be either:
    • find: standard find query.
    • aggregate: aggregation pipeline (cannot use $out).
    • Example: has_relation(Repository:_, String:parent, Organization:_) has variables in the first and third arguments.
  • dbs:
    • connection_string must be a valid MongoDB connection URI or an environment variable: connection_string: $ENV_VAR_NAME.

​
Comma-separated Values (CSV)

version: 1
source: csv
facts:
  has_relation(Repository:_, String:parent, Organization:_):
    fields:
      - name: repository
      - name: organization
    path: /path/to/has_relation.csv
Configuration details:
  • version: must be 1.
  • source: must be csv.
  • facts:
    • Fact type uses positional variables (_) that map to query columns in order.
    • fields: must match the CSV header exactly.
      • Order must match positional arguments in the fact type.
    • path: local path to CSV file.
    • Example: has_relation(Repository:_, String:parent, Organization:_) has variables in the first and third arguments.

​
Oso Sync limitations

  1. At most one Oso Sync command should be run at a time for a given environment. If multiple Oso Sync commands are run in parallel for an environment, you may see HTTP 429 errors.
  2. The maximum size of the application data per fact type is 10GB. To synchronize larger data sets, you may consider “sharding” a single fact type across multiple fact type definitions in the YAML configuration by substituting a concrete value for one or more of the arguments. Before: 
    has_relation(Repository:_, String:_, Organization:_): ...
    After: 
    has_relation(Repository:_, String:parent, Organization:_): ...
has_relation(Repository:_, String:child, Organization:_): ...
  3. The diff may include transient false positives due to our comparing a point-in-time snapshot of your database to Oso Cloud, which continues to receive changes. Transient false positives should not appear on successive invocations of Oso Sync and do not indicate issues with how your application updates facts in Oso Cloud.

​
Docker

We publish a wrapped up version of the CLI for Oso Sync at public.ecr.aws/osohq/reconcile:latest. To use it, build your own image on top of this using a Dockerfile like this: 
FROM public.ecr.aws/osohq/reconcile:latest
ARG CONFIG_PATH
RUN test -n "$CONFIG_PATH" || (echo "CONFIG_PATH argument must be set to path of your reconcile.yaml" && false)
WORKDIR /app
COPY $CONFIG_PATH /app/config.yaml
ENTRYPOINT ["/app/reconcile", "reconcile", "/app/config.yaml"]
Build it with: docker build -t reconcile-tool -f reconcile-tool.Dockerfile --build-arg="CONFIG_PATH=./reconcile.yaml" --platform linux/amd64 ..