Feed Proxy Endpoints

Creatives cannot load data from external URLs directly due to cross-origin restrictions. The feed proxy solves this by fetching external XML, JSON, or CSV data feeds through Cavai's domain, adding caching, filtering, and field remapping.

The proxy provides three endpoints: Remapper v2 (recommended), Remapper v1 (legacy), and a raw proxy.

Remapper v2

GET /proxy/remapper/v2

This request accepts multiple options as GET params.

&url: Required. The URL to proxy data from
&fields: Required. List of fields that are fetched. At least link and name have to be specified. For example, &field[name]=city will map { city: 'Barcelona', country: 'Italy' } to { name: 'Barcelona' }. See Fields section for more details
&default[currency]: Optional. Default currency to return with remapped data, in case data doesn't have currency included and it can't be determined from amount
&filter: Optional. Filters to apply to returned data. For example, &filter[category]=Hodetelefoner will return only rows where category is Hodetelefoner. See Advanced filtering section for more details
&sort_order: Optional. Sorting order asc or desc
&sort_key: Optional. Key to sort by. Sorting is applied after mapping, so output field name should be used
&limit: Optional. How many rows from the URL to return. It is applied after sorting, so &sort_key=amount&sort_order=desc&limit=1 will return only the most expensive item
&format: Optional. Raw data format, accepts xml, csv and json. Default is json
&delimiter: Optional. If the automatic delimiter detection fails for CSV-format, the delimiter can be forced with eg. |, \t
&root_path: Optional. If the item list in data source is nested somewhere, this parameter must be used to specify the path to the list, for example if input data looks like { shop: { sale: { items: [...] } } }, &root_path=shop.sale.items should be added to the request
&cache_time: Optional. Time for how long we keep data in cache before updating it. Defaults to 1 hour

Fields

List of all remappable fields: image, link, currency, name, description, amount, list_amount, location, event_date, event_time, part_of, custom_a, custom_b, custom_c, custom_d, custom_e, custom_f, custom_g, custom_h.

Usage:

&fields[image]: Product image to show in slider
&fields[link]: Product link, supports templating (See URL templating section)
&fields[currency]: Product currency
&fields[name]: Product name
&fields[description]: Product description

URL templating

URL templating allows to create a link using a template (template link) for the path to the product URL (path). It is meant to be used for tracking URLs.

Example:

// Element from external data source
{ movie_url: 'https://netflix.com/movies/666' }

// Part of query
'field[link]=' + encodeURIComponent('${movie_url}&utm-source=cavai_ad')

// Remapped element
{ link: 'https://netflix.com/movies/666?utm-source=cavai_ad' }

Rules:

Templating is allowed only inside the field link.
Template link should always be URL encoded. For example, the link https://track.test/?url=${ProductURL} should be URL encoded to https%3A%2F%2Ftrack.test%2F%3Furl%3D%24%7BProductURL%7D.
A path to the product URL is indicated by the dollar sign and curly braces: ${path}.
Depending on the position of path inside the template link, different rules apply:
- If path is in the begining of template link (${ProductURL}&utm_source=cavai_ad), delimiters for query string parameters will be automatically fixed (as product URL itself can contain query string parameters), so you can use any delimiters in the query.
- If path is in the middle or at the end of template link (https://track.test/?url=${PageURL}&utm=track), pay attention to use the right delimiters and note that the query string parameters that go after the path are part of the external URL (template link), not of the product URL.

Advanced filtering

Filtering items using operators.

Examples:

Find all items with a name containing new: &filter[name][HAS]=new.
Find all items with weight between 100 and 500: &filter[weight][GT]=100&filter[weight][LT]=500.
Find all items which are either red or rounded: &filter[OR][0][color]=red&filter[OR][1][shape]=rounded).
Find all items which are either red and square, or blue and rounded: &filter[OR][0][color]=red&filter[OR][0][shape]=square&filter[OR][1][color]=blue&filter[OR][1][shape]=rounded.

Rules & limitations:

Logical operators are placed before the field name: &filter[NOT][category]=phone. All other operators are placed after the field name: &filter[product][HAS]=5G.
To preserve backward compatibility, omission of an operator after a field name is interpreted as EQ if there is a single value, or as IN if there's an array of values. For example, &filter[color]=red is identical to &filter[color][EQ]=red. And &filter[color]=red&filter[color]=blue is identical to &filter[color][IN]=red&filter[color][IN]=blue.
If operator name is not recognized, it is interpreted as a field name: [cat] turns { cat: { breed: 'sphynx' } } into { breed: 'sphynx' }.
Navigation operators allow multi-level navigation both using paths and chaining. For example, [cat.breed] is identical to [cat][breed] and navigates to sphynx from the previous sample item. So, if we want to find all sphynx cats, we can write the following filter: filter[cat.breed]=sphynx.
The query depth is limited to 6 by Adonis' URL parser.
Fields of a filter object are applied using the logic of AND operator. These queries are identical: filter[type]=cat&filter[breed]=sphynx == filter[AND][0][type]=cat&filter[AND][1][breed]=sphynx or, after Adonis' parser: { type: 'cat', breed: 'sphynx' } == { AND: [{ type: 'cat' }, { breed: 'sphynx' }] }

Logical operators:

const items = [
    { color: 'black', name: 'Bobby' },
    { color: 'black', name: 'Musti' },
    { color: 'white', name: 'Mirri' },
]

OP	Value type	Matches if	Example	Filtered items
OR	Array	Any of conditions is true	`&filter[OR][0][color]=white&filter[OR][1][name]=Musti`	`[{ color: 'black', name: 'Musti' }, { color: 'white', name: 'Mirri' }]`
NOR	Array	No condition is true	`&filter[NOR][0][color]=white&filter[NOR][1][name]=Musti`	`[{ color: 'black', name: 'Bobby' }]`
AND	Array	All conditions are true	`&filter[AND][0][color]=black&filter[AND][1][name]=Musti`	`[{ color: 'black', name: 'Musti' }]`
NOT	Object	Filter doesn't match	`&filter[NOT][color]=black`	`[{ color: 'white', name: 'Mirri' }]`

Comparison operators:

const items = [
    { sum: 2 },
    { sum: 3 },
    { sum: 5 },
]

OP	Value type	Matches if	Example	Filtered items
EQ	String, Number	Is equal to value	`&filter[sum][EQ]=5`	`[{ sum: 5 }]`
NE	String, Number	Is not equal to value	`&filter[sum][NE]=3`	`[{ sum: 2 }, { sum: 5 }]`
IN	Array	Is in array of values	`&filter[sum][IN]=1&filter[sum][IN]=2&filter[sum][IN]=3`	`[{ sum: 2 }, { sum: 3 }]`
NIN	Array	Is not in array of values	`&filter[sum][NIN]=1&filter[sum][NIN]=2&filter[sum][NIN]=3`	`[{ sum: 5 }]`
LT	Number	Is less than value	`&filter[sum][LT]=3`	`[{ sum: 2 }]`
LTE	Number	Is less than or equal to value	`&filter[sum][LTE]=5`	`[{ sum: 2 }, { sum: 3 }, { sum: 5 }]`
GT	Number	Is greater than value	`&filter[sum][GT]=3`	`[{ sum: 5 }]`
GTE	Number	Is greater than or equal to value	`&filter[sum][GTE]=3`	`[{ sum: 3 }, { sum: 5 }]`

Existence operators:

const items = [
    { id: 1, name: 'Jane' },
    { id: 2, name: '' },
    { id: 3 },
]

OP	Value type	Matches if	Example	Filtered items
EXISTS	Boolean	Field exists	`&filter[name][EXISTS]=1`	`[{ id: 1, name: 'Jane' }, { id: 2, name: '' }]`
EMPTY	Boolean	Is empty	`&filter[name][EMPTY]=1`	`[{ id: 2, name: '' }]`
FULL	Boolean	Is not empty	`&filter[name][FULL]=1`	`[{ id: 1, name: 'Jane' }]`

Array operators:

work only with arrays.

const items = [
    { scores: [8,  9, 10] },
    { scores: [5, 10    ] },
    { scores: [5,  6,  8] },
]

OP	Value type	Matches if	Example	Filtered items
EVERY	Array	Has every array element	`&filter[scores][EVERY]=9&filter[scores][EVERY]=10`	`[{ scores: [8, 9, 10] }]`
SOME	Array	Has some array element	`&filter[scores][SOME]=9&filter[scores][SOME]=10`	`[{ scores: [8, 9, 10] }, { scores: [5, 10] }]`
NONE	Array	Has no array element	`&filter[scores][NONE]=9&filter[scores][NONE]=10`	`[{ scores: [5, 6, 8] }]`
LENGTH	Number	The length equals value	`&filter[scores][LENGTH]=2`	`[{ scores: [5, 10] }]`

String operators:

are case-sensitive.

const items = [
    { name: 'Mordor'},
    { name: 'Dora' },
    { name: 'The Lodge Dormitory' }
]

OP	Value type	Matches if	Example	Filtered items
HAS	String	Contains substring	`&filter[name][HAS]=Dor`	`[{ name: 'Mordor' }, { name: 'Dora' }, { name: 'The Lodge Dormitory' }]`

Remapper v1

GET /proxy/remapper/v1

This request accepts multiple options as GET params

&url=: The URL to proxy data from
&format=: Raw data format, accepts xml and csv. Default is json
&limit=: Optional field to limit returned data rows
&filter=: Filters to apply to returned data. For example &filter[category]=Hodetelefoner will return only rows where category is Hodetelefoner
&cache_time=: Time for how long we keep data in cache before updating it. Defaults to 1 hour
&delimiter: If the automatic delimiter detection fails for CSV-format, the delimiter can be forced with eg. "|", "\t"
&sort_order: Sorting order asc or desc
&sort_key: Key to sort by
&root_path: Data to be parsed must be array. In case of nested objects there's possibility to use this as root path to array of data
&default[currency]: Default currency to return with remapped data. In case data doesn't have currency included

There's possibility to remap data field by field

Usage:

&fields[image]: Product image to show in slider
&fields[link]: Product link
&fields[currency]: Product currency
&fields[name]: Product name
&fields[description]: Product description

... etc

Proxy

Pure proxy with some additional filtering

GET /proxy/datafeed?format=xml&url=example.com/

This request accepts multiple options as GET params

&url=: The URL to proxy data from
&format=: Raw data format, accepts xml and facebook
&limit=: Optional field to limit returned data rows
&filter=: Filters to apply to returned data. For example &filter[category]=Hodetelefoner will return only rows where category is Hodetelefoner
&cache_time=: Time for how long we keep data in cache before updating it. Defaults to 1 hour
&generate_urls=: Allows to generate URLs based on field. That's just for testing and debugging before adding URL into the final payload. For example &generate_urls=category will generate all possible URLs by different categories

Example of full URL: https://delivery-0.cavai.com/proxy/datafeed?limit=10&format=xml&filter[category]=Hodetelefoner&url=https://files.channable.com/YohLVYCTmhJ75ug861eufA==.txt

This URL will proxy data from https://files.channable.com/YohLVYCTmhJ75ug861eufA==.txt, limit it to 10 rows and return only data which has category set to Hodetelefoner

Feed Proxy Endpoints ​

Remapper v2 ​

Fields ​

Usage: ​

URL templating ​

Advanced filtering ​

Examples: ​

Rules & limitations: ​

Logical operators: ​

Comparison operators: ​

Existence operators: ​

Array operators: ​

String operators: ​

Remapper v1 ​

Proxy ​

Feed Proxy Endpoints

Remapper v2

Fields

Usage:

URL templating

Advanced filtering

Examples:

Rules & limitations:

Logical operators:

Comparison operators:

Existence operators:

Array operators:

String operators:

Remapper v1

Proxy