Skip to main content
MoEngage automates imports of users, events, and auxiliary data from files you drop in an Amazon S3 bucket or an SFTP server. You prepare your files, configure the connection and column mapping once, and MoEngage picks up new files on a one-time or periodic schedule. Select your source in any tabbed section below and the page stays on that source throughout.

Types of Imports

MoEngage can import the following from your files:
  • Registered Users: Users who are already registered on MoEngage.
  • Anonymous Users: Users who are not yet registered on MoEngage.
  • Events (Standard and User-Defined): Standard events such as Campaign Interaction Events, as well as your own user-defined events.
  • Auxiliary Data: Supplementary user data ingested temporarily. Auxiliary Data has its own setup. See Auxiliary Data.

Prepare Your Files

Supported File Types

MoEngage supports CSV and NDJSON files, in UTF-8 encoding.

Naming Conventions

Name your files so that MoEngage can identify the import type and the run date:
Give every file a unique name. MoEngage skips any file name it has already processed, whether the previous import succeeded, partially succeeded, or failed.
A file name has two parts: <import type>_<date time format>.<file extension>.
  • Registered users: prefix registered_user_data_, for example, registered_user_data_01311997.csv.
  • Anonymous users: prefix anonymous_user_data_, for example, anonymous_user_data_jan_2022.csv.
  • Events: prefix <event name>_, for example, for the event “Purchase Summary”, Purchase_Summary_31011997.csv.
MoEngage also supports custom file names following <custom name>_<date time format>.<file extension>, where the custom name uses letters, numbers, spaces, and underscores. The <date time format> suffix represents the day the import runs. For example, if an import runs on 7 January 2022 and the selected format is ddmmyyyy, MoEngage looks for files ending in 07012022.csv or 07012022.json. Configure the format on the dashboard during setup. Supported formats:
Do not use moe_ as a prefix for events, event attributes, or user attributes. It is a reserved system prefix.
File extensions:
  • Without encryption: .csv, .json
  • With PGP encryption: .csv.asc, .json.asc, .csv.pgp, .json.pgp

File Structure

Structure your files according to the file type you import:
  • The first row must be the column names (treated as the header row).
  • No duplicate column names (names are case-sensitive) and no missing column names.
  • Only the comma (,) is supported as a delimiter.
  • Avoid empty rows.
Sample files: Registered Users, Anonymous Users, Events, Mapping File.
MoEngage does not support the | (pipe) character in non-array columns. Ensure String, Numeric, and Boolean columns do not contain this character.

Folder Structure

Place all files into a single folder and configure the folder path during setup. MoEngage does not look inside subfolders or the root folder.

Array Support (CSV)

  • Simple arrays are supported. Nested and associated arrays are not supported.
  • The supported delimiter in the CSV is the pipe (|). For example: [1|2|3].
  • The supported data types are Array String and Array Double (numeric). Select the type in the Map column to attribute list during column mapping.
  • All elements in an array must be homogeneous, either all integer/double or all string. If they are not, MoEngage attempts to typecast each element to the declared type and stores the result.
  • Array typecasting: If an array ["1" | 2 | "abc"] is mapped as Array Integer, MoEngage ingests [1, 2], because "abc" cannot be cast to an integer and is dropped. For more information, see typecasting in MoEngage.

Limits

File imports have the following limits: Rate limits (defaults, contact MoEngage to increase): The hourly rate is the maximum ingested per hour; the daily rate is the total cap per day. When a rate limit is breached, you receive a Rate limit breached alert on the Data Imports dashboard.
Avoid scheduling a very large number of files at the same time. Doing so can create a processing backlog and delay ingestion, even when each file is small. Stagger or batch your scheduled imports over time instead of queuing a large number of files together.

Required Access and Connection

Grant MoEngage access to your source before you create the import:
Grant MoEngage read access to your S3 bucket and set up a connection in the App Marketplace. For the IAM permissions and connection setup, see Set Up Authentication, Grant Permissions, and Connect Amazon S3 in the App Marketplace.

Set Up the Import

Set up your import in three steps: configure the connection and file format, map your columns to MoEngage attributes, and select the import frequency.
Contact your MoEngage Customer Success team to enable the Data menu on the left navigation if it is not visible.
On the MoEngage sidebar, go to Data > Data imports, click + Import, and select Users or Events. Then select your source tile and click Continue.

Step 1: Configure the Connection and File Format

Select whether to import Registered users or Anonymous users (for User imports), or select the event to import (for Event imports). Registered and anonymous imports expect different file names; see Naming Conventions. For events, your file names must begin with the event name of the chosen event. For example, the event “App/Site Opened” has the event name MOE_APP_OPENED, so file names begin with MOE_APP_OPENED_. To create a new event, click + Create event and enter a unique name; new events appear in your app only after the first successful import. Next, configure the connection and folder:
  1. In the S3 connection list, select your connection (or click + Add connection).
  2. In the File type list, select CSV or JSON.
  3. In the Folder path box, enter your directory path. Do not include the bucket name or trailing slashes, because MoEngage resolves the bucket name from your marketplace connection.
  4. To import PGP-encrypted files, select Is your file encrypted? and provide your private Decryption Key, optional public Signing Key, and Key passphrase.
S3 connection, file type, folder path, and encryption options
Enter a unique Import name to identify this import on the dashboard.
PGP (Pretty Good Privacy) lets you encrypt sensitive files before importing them. MoEngage supports PGP encryption for S3 and SFTP file imports.Prerequisites:
  • PGP key pair: A public key (to encrypt the file) and a private key (to decrypt it). Never share the private key.
  • Signing key pair: A public key (to sign the file, required for MoEngage SFTP integration) and a private key (used internally to verify the signature).
The following steps use GnuPG, a free implementation of the PGP standard.Step 1. Generate a key pair. Generate a PGP key pair using a tool such as GnuPG (gpg --full-generate-key), selecting RSA and a 1024-bit key size.
  • MoEngage supports file encryption using RSA keys of up to 1024 bits in length. Larger RSA keys, such as 4096-bit, are not supported and fail during decryption. Generate your key pair with a smaller RSA key size.
  • Turn off compression during encryption.
Step 2. Encrypt the file. Encrypt the file with your public key, then upload it to your S3 or SFTP folder. The --sign option signs the file while encrypting it.
Step 3. Add the keys in the MoEngage UI. Enter your private key in the Decryption Key field and your passphrase in the Key Passphrase field during import setup.Step 4. Decrypt a file. Decryption requires your private key and its passphrase. In your PGP software, select the encrypted file and enter your passphrase to produce the unencrypted file. Python users can use the python-gnupg package:
Step 5. Verify the file’s signature (optional). If you signed the file, add the public signing key on the MoEngage dashboard. MoEngage verifies the signature on the file in your S3 or SFTP folder. When verifying programmatically, a matched signature populates status.signature_id:

File Configurations

Select the DateTime format present in your file name (your file name must end with _<date time format>.csv). Ensure at least one matching file exists in the folder, then click Fetch file(s). If the connection and format are correct, MoEngage shows a preview of the top 5 rows. If MoEngage cannot fetch the file, you may see:
  • Given S3 Access Key is not valid: Verify your S3 marketplace connection details.
  • Please check the folder path and try again: Verify the folder path and that compatible files exist in it.
  • Found zero data rows in the file: The file has no rows after the header. MoEngage expects at least 2 rows (1 header + 1 data row).
Click Next to continue to mapping.

Step 2: Map Your Columns to MoEngage Attributes

Map each file column to a MoEngage attribute. For every column you see the column name with a sample value, the Map attribute selector (pick the data type, and for DateTime columns the format), and a Skip action. To create a new attribute, click + Create attribute. You can also click + Add Field to enter a mapping manually. Mandatory mappings depend on the import type:
  • User imports (Registered): map your identifier column to User ID.
  • User imports (Anonymous): map your identifier column to Anonymous ID.
  • Event imports: map your identifier column to User ID, and map your timestamp column to Event Time.
For events, map either Event Time (UTC) or Event Time (App Timezone). If you choose Event Time (UTC), the event time is converted to the timezone configured in your MoEngage dashboard settings.
Once a mandatory mapping is marked, you can no longer skip that column.

Mapping Files

Optionally, auto-map your columns by uploading a mapping file. Click Upload mapping file at the top-right of the mapping table and select your file. Mappings are auto-configured; a modal lets you create any attributes referenced in the mapping file that don’t yet exist. Columns in the mapping file but not in your file are ignored; columns in your file but missing from the mapping file are left blank for manual mapping. For a fresh import, MoEngage provides a sample mapping file you can use as a reference. If you duplicate an existing import, its mapping file is pre-attached and available to download. If a mapping file maps a column to an attribute that doesn’t exist in MoEngage, the mapping is left blank, create the attribute from the UI first, then map the column to it. A mapping file contains the mappings between each source column and a MoEngage attribute, along with the data type of the column. The file must be in JSON format. Instead of mapping columns one by one on the dashboard, you can upload a mapping file to automate the mapping.
For each column, provide the following fields:
  1. column (required): The column name from the source file. For Level 2 keys in a JSON file, use dot notation (key1.key2).
  2. moe_attr (required): The MoEngage attribute to map the column to. Ensure each column maps to a unique moe_attr.
  3. type (optional): The data type of the column. See the supported types below.
  4. datetime_format (optional): The date-time format. Mandatory for DateTime fields only.
  5. is_skipped (optional): A boolean field. Any column marked true is skipped during import.

Standard User Attributes for Reference

Map your source columns to MoEngage standard user attributes using the keys below. For the exhaustive list, refer to your Data Management dashboard.
Track standard string attributes with the correct data type. For example, if First Name (u_fn) is ingested as a number or an array instead of a string, the sample users on the Create segment page fail to load with a 500 error (“There seems to be an error”). To fix this, pin the attribute’s data type to String on the Data Management dashboard and re-send the corrected data for the affected users.

Supported Attribute Types

MoEngage does not support the | (pipe) character in non-array type columns. Ensure your String, Numeric, and Boolean columns do not contain this character.

Reserved Keywords for User Attributes

MoEngage reserves the following keys. Do not use them when you map or track user attributes.
  • USER_ATTRIBUTE_UNIQUE_ID
  • USER_ATTRIBUTE_USER_EMAIL
  • USER_ATTRIBUTE_USER_MOBILE
  • USER_ATTRIBUTE_USER_NAME
  • USER_ATTRIBUTE_USER_GENDER
  • USER_ATTRIBUTE_USER_FIRST_NAME
  • USER_ATTRIBUTE_USER_LAST_NAME
  • USER_ATTRIBUTE_USER_BDAY
  • USER_ATTRIBUTE_NOTIFICATION_PREF
  • USER_ATTRIBUTE_OLD_ID
  • MOE_TIME_FORMAT
  • MOE_TIME_TIMEZONE
  • USER_ATTRIBUTE_DND_START_TIME
  • USER_ATTRIBUTE_DND_END_TIME
  • MOE_GAID
  • INSTALL
  • UPDATE
  • MOE_ISLAT
  • status
  • user_id
  • source

Portfolio Support (Project-Level Routing)

You can route imported users and events to specific projects in your MoEngage portfolio workspace using column mapping. Map your identifier column (such as brand_name or app_id) to the MoEngage attribute moe_project_name. Values must match your MoEngage project names exactly (matching is case-sensitive).
  • Successful routing: If the value matches a project in your portfolio, MoEngage ingests the user or event at that project level.
  • Fallback: If the mapping is missing, blank, or does not match a project name, MoEngage ingests the data at your global portfolio level.

Send Import Notifications

Turn on notifications and select up to 10 email recipients to be alerted when an import is created, succeeds, or fails. When your mappings are complete, click Next.

Step 3: Select the Import Frequency

Define when MoEngage retrieves your files:
  • One-Time Imports: Run immediately or schedule for a future date and time.
  • Periodic Imports: Run hourly, daily, weekly, or monthly, with intervals and advanced configurations.
For each run, MoEngage fetches files matching the configured DateTime format, so ensure your file names are correct. Optionally, set the import to end after a number of occurrences or on a specific date. Click Done.
Channel reachability and custom segments for imported users are available within 2 hours of import completion. Plan your campaigns accordingly.

Duplicate Imports

You can configure only one unique import at a time. An import is a duplicate when all of these match an existing active import:
  1. Import type: Users or Events.
  2. Import sub-type: Event name, or Registered / Anonymous.
  3. S3 folder path.
  4. Bucket name.
If any one parameter differs, the import is unique.

Import Failure Policy

MoEngage retries a failed file automatically, up to 30 attempts, before marking it permanently FAILED. You can configure an email alert on failure during setup. A file-level failure does not stop the schedule, future scheduled runs continue independently.

Supported Datetime Formats

Use these in the datetime_format field of your mapping file, or when configuring DateTime columns during mapping.

Manage Your Imports

To monitor your imports, understand import statuses, or trigger a run manually or via API, see How Imports Work on the Imports Overview.

Frequently Asked Questions

Click the ellipsis on the right and click View to look up the Import details. Hover over the Failed Status to learn the reason.
In such cases, the new data will still be added to the archived segment. You can unarchive the segment as required.
Once an import process starts, it can’t be stopped midway. This is because the data goes through several steps, and interrupting it could lead to incomplete or inconsistent results. It’s best to let the current import finish.
Yes, you can stop future scheduled imports from running automatically. To do this, find the import schedule and select the Archive option from the Actions menu on the Data Imports dashboard. This will prevent it from running on its next scheduled time.
If an import appears to be stuck or is taking longer than usual, it’s best to wait. The system has checks in place to handle these situations automatically and retry if necessary. Manually starting the same import again while it’s still processing can cause conflicts and may prevent the original import from completing successfully.
MoEngage retries a failed file automatically, up to 30 attempts, before marking it permanently FAILED. You can also configure an email alert on failure by selecting the notification checkbox during setup.
No. When a file exhausts its processing retries and fails with the Applicable rate limit breached status, MoEngage cannot replay that file on its own, so the periodic schedule does not reprocess it automatically. This retry limit protects the ingestion pipeline from being overloaded. To reprocess the affected files, rename them and re-upload them to your source folder so that MoEngage treats them as new files. To reduce the chance of hitting rate limits, stagger large batches instead of scheduling many files at once (see Limits).
During each scheduled run, MoEngage searches your configured folder path and selects new files based on your file-name prefix and datetime-format configuration. It uniquely identifies files by the combination of folder location and file name, so it processes each file only once.
No. A file that has already been processed is not processed again while its file name and folder location stay the same. If you place the same file in a different folder, an import configured for that new folder will process it again.
No. Set up user imports and event imports as separate tasks. Each source file must contain only user attributes or only event records.
NDJSON (Newline-Delimited JSON) means each line in your .json file is a complete JSON object separated by a newline. This format lets MoEngage process large JSON datasets line by line for scalable, robust imports.
No. MoEngage does not support importing compressed files.
Changing the file type removes the existing preview and re-fetches the available files for the new file type. You must select the file to preview again and reconfigure your mappings based on the new file’s structure.