Snapshot
When importing data to Stae, we offer two different methods (visual and code) to transform your data to Stae's official data standards. This tutorial covers the Importer's utility options for the code method. We'll provide sample code to get you started in transforming your data sets.
This Tutorial is for Admins Only
Only accounts with Admin Access can import data.
To start, find the Building Permits dataset from your city's open data portal. If one doesn't exist, you can use another common dataset like 311 issues. For non-admins, if you have a dataset you'd like to upload to Stae, get in touch with us at we@stae.co
At the end of this tutorial, you’ll be able to
- Use the Importer utilities to perform various transformation functions
1. Navigate to the Importer
To navigate to a data source's Importer, select the Sources button and then find the data source from the list. For information on creating your own data source, visit the Creating a Data Source tutorial.
Sources page in Jersey City
Once on the source's page, navigate to the Importer portion of the page and select Create Importer. The Importer can accept a variety of formats as either a file or an API: JSON, CSV, XML, Excel, SHP, GTFS, and GTFS-RT.
Select "Create Importer" to build a new importer
You can import a variety of formats as either a file or an API: JSON, CSV, XML, Excel, SHP, GTFS, and GTFS-RT
Depending on the file type, you may have an additional step that prompts you to target the fields you'd like to import. In most cases, you'll want to import all the data. Select "Features > All" from the dropdown menu.
Step 2 allows you to target what portion of data you'd like to transform.
Once you're on step 3, you can begin mapping your fields. The following section will show you how to use different utility functions in the Code tab to transform your data.
2. Types of Utility Functions
Here's the full list of importer utilities on GitHub.
Undefined Responses
Utility functions will return "Undefined" when given null or invalid values when appropriate.
ID Generator - util.guid
Stringifies all provided values, turning them into a deterministic GUID. Null values are ignored.
Slug Generator - util.slug
Stringifies all provided values, turning them into a lowercase slug. Null values are ignored.
Normalization - util.normalize
This function trims, lowercases, and removes redundant whitespace from the given value.
Capitalization - util.capitalize
This function capitalizes the first word in the given value.
Capitalize all words - util.capitalize.words
This function capitalizes every word in the given value.
Phone number normalization - util.phone
This function normalizes any phone number in the world. Country is any three letter country code, defaulting to 'USA'. If the number includes a country code, you do not need to provide a country argument.
Unit conversion - util.convert
This function exposes convert-units, with additional handling of null and invalid values. For a full list of convertible units, see this page.
Moment JavaScript wrapper - util.date
This function is a wrapper around moment.js moment.tz(date, timezone)
that handles null and invalid values.
Parse numbers - util.number
This function is a wrapper around number parsing that handles null and invalid values.
Sea coordinates - util.geo.isSea
This function returns true if the coordinate is in the sea. Value can be either [ lon, lat ] or a GeoJSON point.
Convert GeoJSON geometry to Multi - util.geo.multi
This function converts any given GeoJSON geometry to a Multi version. Values that are already a Multi geometry will be returned with no modifications.
Convert WKT to GeoJSON - util.geo.wk2JSON
This function converts Well-Known-Text to GeoJSON - supports WKT/WKB/EWKT/EWKB/TWKB.
Convert address to GeoJSON - util.geo.locate
This function converts any address to a GeoJSON point.
Convert intersection to GeoJSON - util.geo.locate
This function converts any intersection to a GeoJSON point.
Get timezone to GeoJSON Point - util.geo.tz
This function returns the timezone for a given point. The value returned can be used as the timezone for util.date
. The function can handle either [ lon, lat ]
or a GeoJSON Point.
Additional Utilities
Stae Inspired
“In God we trust, everyone else bring data.” Michael Bloomberg, Bloomberg Philanthropies
Have any questions or running into issues with this feature?
Reach us at: support@stae.co
Comments
0 comments
Please sign in to leave a comment.