CouchDB
Overview
Section titled “Overview”Signals reads customer and event documents straight from a CouchDB instance on a schedule and sends the columns you nominate to your ad platforms.
Reading it directly means there is no file job to maintain and nothing to go stale between runs. Signals reads only; nothing is written back.
CouchDB stores documents in databases rather than rows in tables, and has no schema of its own. What Signals expects is therefore entirely down to what your documents contain.
You can point it at a database, or at a query. The query path is what you want when the shape Signals needs does not match anything you already have.
Everything CouchDB does happens over HTTP. The host and the port are the beginning of a URL, a database is a path underneath it, and a document is a path under that, so any read described on this page can be reproduced with curl from anywhere that reaches the server.
No part of this involves SQL. A query is a Mango selector: a JSON document naming the fields to match and the values to match them against, sent to _find on the database. The fields array beside it picks which parts of each document come back, and picking is the whole of what it does, because Mango has no means of renaming a field on the way out. A _find request sent by hand also stops at 25 documents unless it sets limit, so a selector matching a hundred thousand documents answers with 25 of them and nothing to mark the difference.
Supported destinations
Section titled “Supported destinations”Destinations supported by CouchDB
| Category | Supported |
|---|---|
| Offline Conversions | |
| Lead Conversions | |
| Custom Audience |
Authentication
Section titled “Authentication”In Studio, open Sources, find the Database category, and click the CouchDB tile.
| Field | What it is | Where to find it |
|---|---|---|
| Authentication Name | Your own label for this set of credentials. | You choose it. Name it for the database it belongs to. |
| Username | The database user Signals connects as. | Created in the database by your administrator. |
| Password | The password for that user. | Set when the user was created. |
Signing in to the server and being let into a database are two decisions, and the second one is stored inside the database. Every database holds a _security document with an admins block and a members block, each listing names and roles:
{ "admins": { "names": ["dbadmin"], "roles": [] }, "members": { "names": ["datahash"], "roles": [] }}As soon as a single member name or role is present, only an authenticated user matching one of them may read documents from that database, or so much as ask the database for itself. An account left out of the list is a valid account with a valid password that this database declines, and putting its name in members is the entire fix.
An empty members block means the reverse of locked down: with no names and no roles listed at all, any user can read the database and write regular documents into it. Databases created on CouchDB 3.x carry the _admin role by default for that reason.
Do not test with the server admin. CouchDB 3.0 made a server admin mandatory, printing an error and exiting at startup where none is configured, so one always exists and handing it over is the path of least resistance. A server admin holds _admin and is admitted to every database whatever its _security says, so a read that succeeds as the admin confirms the host, the port and the database name while proving nothing about what the read-only account can see.
Configuration
Section titled “Configuration”Instance Setup
Section titled “Instance Setup”Select the Integration Type, which is the kind of data your records hold. Offline Events is the common choice. Give the instance a name, then choose the table path or the query path. On either path you enter the Database Name, Host Name and Query Service Port, and then either the Collection Name or a query.
The integration type decides the columns Signals expects and which destinations the connection can feed, so it is worth settling before anyone models the data. It also decides which file format you get on the next screen.
Port and what counts as a database name
Section titled “Port and what counts as a database name”5984 is the port a stock CouchDB answers on, and 6984 is the encrypted one, which exists only after enable = true has been set in the [ssl] section of the configuration and a certificate and key have been pointed at. The port is what decides which of the two the connection speaks, rather than a protocol setting sitting beside it: 5984 and 6984 are the difference between http:// and https:// reaching the same server.
Database names are constrained more tightly than a free text field suggests. CouchDB accepts a name matching ^[a-z][a-z0-9_$()+/-]*$ and refuses everything else, which means it begins with a lowercase letter and can never contain a capital. A name typed here with capitals in it corresponds to no database that could have been created in the first place, so it is worth reading the name before investigating anything further.
Table path
Section titled “Table path”Enter the connection details including the database name, then complete the setup. The name is typed rather than picked from a list, so it has to match the database exactly.
Query path
Section titled “Query path”Enter the connection details without the database name, then write the query. Preview the results before finishing, and read the column names in the output rather than just the row count: a query that runs but returns the wrong names will fail silently once the connection is live.
Manage instance
Section titled “Manage instance”One instance reads one table or query for one integration type. To read another, add another instance from the Manage existing instance table. To change an existing one, open it from there, click the edit option in the menu to the top right, update the fields and click Finish.
File format and schema
Section titled “File format and schema”Download the file format from the setup screen and shape the database to it. The column names are what Signals matches on, so a column that has been renamed or cased differently is not recognized and the row it belongs to is dropped rather than reported.
On the query path the names are still whatever the documents hold, since a Mango selector chooses fields and cannot rename them. Reconciling a name that does not match therefore means changing what the documents store, rather than aliasing anything in the query.
Data & identifiers
Section titled “Data & identifiers”Personal identifiers are normalized and hashed before they are sent to a destination.
Best practices
Section titled “Best practices”- Use a dedicated read-only user scoped to the data Datahash reads, rather than an application account.
- Take the query path for the filtering, not for reshaping names. A selector narrows which documents are read and hands back their fields as stored.
- Replicate the database to a copy Datahash reads instead of pointing the connection at the one your application writes to. A CouchDB replication target is a complete database rather than a subordinate of its source, so scanning it on a node of its own keeps the load off the node your application depends on.
- Create a user with read access to the one database.
- Store nested document fields as top-level values with the expected names.
- Clean the data in the documents before Signals reads them: lowercase email, phone in E.164, no stray whitespace. Mango has no function to apply to a value on the way out, so an address saved as
Bob@Example.COMis the exact string that gets hashed. Hashing a badly formatted value produces a hash that will never match. - Rotate the password periodically and update the connection afterwards.
Troubleshooting & FAQ
Section titled “Troubleshooting & FAQ”The connection times out rather than reporting bad credentials. That points at reachability rather than authentication. Confirm the host is exposed, the port is open, and any firewall or allow list permits the connection.
Credentials are rejected. Check the user can read the data from a client of its own first. Most failures here are permissions rather than typos.
The database name is rejected. It is typed rather than selected, so it has to match exactly.
The query previews correctly but nothing is delivered. The field names it returns do not match the expected schema. Rename the fields in the documents to the names in the downloaded file format.
The password is accepted and the database is refused anyway. CouchDB keeps those two answers apart by status code and by one word. An authenticated user who is not a member of the database gets 403 and You are not allowed to access this db., while a request arriving with no authenticated user at all gets 401 and You are not authorized to access this db. The two sentences read almost the same and point at opposite fixes. The 403 says the credentials were fine and the name is missing from members in that database’s _security, and no amount of resetting the password will alter it. Read the status code before reading the sentence.
A selector that returns the right documents takes far too long to do it. A Mango query with no index able to serve it does not fail. CouchDB falls back to the built-in _all_docs index, which its own documentation describes as arbitrarily slow, and reports the fallback as a warning inside an otherwise successful response: No matching index found, create an index to optimize query time. The documents come back correct, so nothing downstream objects, and the cost stays out of sight until the database is large enough that reading every document in it takes real time. Post an index over the fields the selector filters on to _index on the database, then confirm the warning has gone. Sending allow_fallback: false with a query turns the same situation into an immediate error, which is how to establish whether an index is being used at all. That parameter arrived in CouchDB 3.4. On 3.3.x and earlier, Mango rejects any field it does not recognize in a _find body, so the same request comes back complaining about an invalid key and tells you nothing about the index.
It was working and stopped. The password may have been changed or the user disabled. A dedicated service user avoids this.