Importer Utilities

Snapshot

When importing data to Stae, we offer two different methods (visual and code) to transform your data to Stae's official data standards. This tutorial covers the Importer's utility options for the code method. We'll provide sample code to get you started in transforming your data sets.

 

This Tutorial is for Admins Only
Only accounts with Admin Access can import data.
To start, find the Building Permits dataset from your city's open data portal. If one doesn't exist, you can use another common dataset like 311 issues. For non-admins, if you have a dataset you'd like to upload to Stae, get in touch with us at we@stae.co

 

At the end of this tutorial, you’ll be able to

• Use the Importer utilities to perform various transformation functions

 

1. Navigate to the Importer

To navigate to a data source's Importer, select the Sources button and then find the data source from the list. For information on creating your own data source, visit the Creating a Data Source tutorial.

 

Sources page in Jersey City

Once on the source's page, navigate to the Importer portion of the page and select Create Importer. The Importer can accept a variety of formats as either a file or an API: JSON, CSV, XML, Excel, SHP, GTFS, and GTFS-RT.

Select "Create Importer" to build a new importer

 

You can import a variety of formats as either a file or an API: JSON, CSV, XML, Excel, SHP, GTFS, and GTFS-RT

Depending on the file type, you may have an additional step that prompts you to target the fields you'd like to import. In most cases, you'll want to import all the data. Select "Features > All" from the dropdown menu.

Step 2 allows you to target what portion of data you'd like to transform.

Once you're on step 3, you can begin mapping your fields. The following section will show you how to use different utility functions in the Code tab to transform your data.

 

2. Types of Utility Functions

Here's the full list of importer utilities on GitHub.

Undefined Responses

Utility functions will return "Undefined" when given null or invalid values when appropriate.

ID Generator - util.guid

Stringifies all provided values, turning them into a deterministic GUID. Null values are ignored.

 

Slug Generator - util.slug

Stringifies all provided values, turning them into a lowercase slug. Null values are ignored.

 

Normalization - util.normalize

This function trims, lowercases, and removes redundant whitespace from the given value.

Capitalization - util.capitalize

This function capitalizes the first word in the given value.

Capitalize all words - util.capitalize.words

This function capitalizes every word in the given value.

Phone number normalization - util.phone

This function normalizes any phone number in the world. Country is any three letter country code, defaulting to 'USA'. If the number includes a country code, you do not need to provide a country argument.

Unit conversion - util.convert

This function exposes convert-units, with additional handling of null and invalid values. For a full list of convertible units, see this page.

 

Moment JavaScript wrapper - util.date

This function is a wrapper around moment.js moment.tz(date, timezone) that handles null and invalid values.

Parse numbers - util.number

This function is a wrapper around number parsing that handles null and invalid values.

Sea coordinates - util.geo.isSea

This function returns true if the coordinate is in the sea. Value can be either [ lon, lat ] or a GeoJSON point.

Convert GeoJSON geometry to Multi - util.geo.multi

This function converts any given GeoJSON geometry to a Multi version. Values that are already a Multi geometry will be returned with no modifications.

Convert WKT to GeoJSON - util.geo.wk2JSON

This function converts Well-Known-Text to GeoJSON - supports WKT/WKB/EWKT/EWKB/TWKB.

Convert address to GeoJSON - util.geo.locate

This function converts any address to a GeoJSON point.

Convert intersection to GeoJSON - util.geo.locate

This function converts any intersection to a GeoJSON point.

Get timezone to GeoJSON Point - util.geo.tz

This function returns the timezone for a given point. The value returned can be used as the timezone for util.date. The function can handle either [ lon, lat ] or a GeoJSON Point.

Additional Utilities

Stae Inspired

“In God we trust, everyone else bring data.”  Michael Bloomberg, Bloomberg Philanthropies

Have any questions or running into issues with this feature?
Reach us at: support@stae.co