Docs

Data Shim is meant to be a quick and dirty bridge between an API or data source and the API that you actually need. It can be used to manipulate any datasource to fit to your specifications.

You create a data shim by writing a Bash shell snippet. This snippet is assigned a URL that, when called, returns the output of a script snippet.

Tutorials

Examples

Convert HTTP to HTTPS and vice versa

curl -s "http://www.thecocktaildb.com/api/json/v1/1/lookup.php?i=12107" | jq -r .

Run It

Add CORS to an existing API

curl -s "http://samples.openweathermap.org/data/2.5/weather?q=London&appid=b1b15e88fa797225412429c1c50c122a1"

Run It

Convert XML to JSON

curl -s "http://samples.openweathermap.org/data/2.5/weather?q=Tel+Aviv&appid=b1b15e88fa797225412429c1c50c122a1&mode=xml" | xml2json

Run It

Not the best example as this API actually returns JSON be default anyway.

Convert a JSON response to a different structure

curl -s "http://www.thecocktaildb.com/api/json/v1/1/lookup.php?i=12107" | jq -r '.drinks[0] | { name:.strDrink, glass:.strGlass}'

Run It

Manipulate HTML

curl -s "https://news.ycombinator.com/" | pup 'table table tr:nth-last-of-type(n+2) td.title a attr{href}' | grep ^http | jo -a | jq -r .

Run It

Using Parameters

You can use parameters to dynamically change the Shim. Reference the parameter as bash variable inside a shim.

Below are a few examples of a using a variable to change the URL of an image.

Return Image

curl -s "$imgURL"

Run It

Get Image Dimensions

identify <(curl -s "$imgURL") | awk '{print $3}' | jo -a | jq '. | {size: .[0]}'

Run It

Crop Image

curl -s "$imgURL" | convert - -gravity North -crop 100x100+0+0 +repage -

Run It

Upload Static Data and Operate On It

There is a field for uploading static data for the Shim to operate on.

See the CSV example below.

Extract Column Names From Static CSV

csvcut -n <(echo "$file") | awk '{print $2}' | jo -a | jq -r .

Run It

Run SQL Query on CSV Data

echo "$file" | csvsql --query 'SELECT "Index"," ""List Price ($)""" FROM stdin WHERE "Beds" > 3;'

Run It

Generate Data

jo -p name=Jane data[]="some data" point[]="some other data" geo.lat=10 geo.lon=20

Run It

Generate Random Data

jo -p name="$(chance name)" email="$(chance email)" user_id="$(chance guid)"

Run It

Pretty Print XML

curl -s "http://my.mobfox.com/request.php?rt=api-fetchip&r_type=banner&s=fe96717d9875b9da4339ea5367eff1ec&adspace_width=320&adspace_height=50&o_iosadvid=1976F519-26D0-4428-9891-3133253A453F&u=Mozilla%2F5.0%20(iPhone%3B%20CPU%20iPhone%20OS%208_0%20like%20Mac%20OS%20X)%20AppleWebKit%2F600.1.3%20(KHTML%2C%20like%20Gecko)%20Version%2F8.0%20Mobile%2F12A4345d%20Safari%2F600.1.4&r_resp=xml" | xmllint --format -

Run It

Pretty Print JSON

curl -s "http://my.mobfox.com/request.php?rt=api-fetchip&r_type=banner&s=fe96717d9875b9da4339ea5367eff1ec&adspace_width=320&adspace_height=50&o_iosadvid=1976F519-26D0-4428-9891-3133253A453F&u=Mozilla%2F5.0%20(iPhone%3B%20CPU%20iPhone%20OS%208_0%20like%20Mac%20OS%20X)%20AppleWebKit%2F600.1.3%20(KHTML%2C%20like%20Gecko)%20Version%2F8.0%20Mobile%2F12A4345d%20Safari%2F600.1.4&r_resp=json" | jq -r .

Run It

Code Highlighting

echo "$file" | pygmentize -O full,style=monokai -f html -l javascript

Run It