Audience definition overview

The module created to export lists of different user types. These exports are available in view form or in CSV format. The Exports are used on different third-party tools like Hubspot, and/or for mass-mailing.

This module is mostly used to enable our sales and ad targeting efforts.

This module is only accessible with an Admin level account.

The export lists available are listed on the admin panel, as well as an explanation of how to filter the exports.

• How is an Audience Definition defined?
  • Audience Definition vs Audience List
  • Class Naming
  • Audience Definition description
• Data Handling
  • Why are Audience Definitions services
  • AbstractAudienceDefinition class
  • Data Chunking and Pagination
• Testing
  • Missing production tables
  • Memory issues

How is an Audience Definition defined?

Audience Definitions themselves are service classes not models, that define organized sets of Member or Subscriber data organized by certain rules.

Examples of this are:

• All active Members
• All newsletter Subscribers
• Inactive paying Members

Audience Definition vs Audience List

Audience Definition are the services that define the rules to filter the data, while Audience List are the actual data that is displayed/exported.

From the user perspective they're exporting Audience Lists, but from the code perspective we're defining Audience Definitions.

Class Naming

The name of an Audience Definition determines the displayed and export file name.

For example, the class AllActiveMembers will be displayed as All Active Members on the Audience List page, as well as, resulting in a All Active Members.csv filename.

IndividualMembersSubscribedToNewsletter will be displayed as Individual Members Subscribed To Newsletter and result in a Individual Members Subscribed To Newsletter.csv filename.

So it becomes important to have a self-explanatory class name to have a good display and export file name.

Audience Definition description

Audience lists can also be parameterized, allowing for more flexibility in the data that is displayed/exported. These parameters are communicated through the description and follow the format %parameter_name%.

For example:

protected string $description = 'Individual, canceled members who have canceled their membership in the past 7 %days%';

This example would convey to the user that the default value is 7 days, however the user could alter the URL and send parameter with days=X to define a different time range for the Audience List.

On the Audience Definition service, the parameter is used to apply said filter:

public function getQuery(): Builder
{
    return Member::query()
        ->scopes(['individual', 'paying'])
        ->with(['country'])
        ->where('deleted_at', '>', now()->subDays((int) ($this->parameters['days'] ?? self::CANCELLED_DAYS_AGO)))
        ->withoutGlobalScopes([SoftDeletingScope::class]);
}

That's why having a good description is very important to define an Audience list's usability.

Data Handling

This module is designed to handle large datasets efficiently by utilizing data chunking and streaming techniques both for display and export functionalities.

Each Audience list must define its own service, which is a class that extends the AbstractAudienceDefinition class.

Why are Audience Definitions services

While we could directly query the Member and Subscribers models, the Audience Definition services are used to encapsulate the logic of complex queries and data transformation.

A good example of this is the AllNonMemberFreeEbookSubscribers that defines a complex query on Subscribers filtering inactive member that subscribed to the free ebook list and are not from a well-known email provider:

public function getQuery(): Builder
{
    /** @var \Illuminate\Database\Eloquent\Builder<\App\Modules\Subscription\Models\Subscriber> $query */
    $query = Subscriber::query()
        ->with(['country'])
        ->doesntHave('activeMemberByEmail')
        ->where('contact_list_id', ContactList::FREE_EBOOK_LIST_ID)
        ->where('subscription_status', SubscriptionStatus::Active);

    foreach (\Illuminate\Support\Facades\Config::array('ixdf_member.well_known_email_providers_domains') as $domain) {
        $query->where('email', 'NOT LIKE', "%{$domain}%");
    }

    return $query;
}

Decoupling the query and data handling from the models allows for better organization and maintainability of the Audience Definition module and the Member and Subscription modules.

AbstractAudienceDefinition class

This service class is the base class for all Audience lists. It's responsible for:

• Converting the class name to the humanized form for display and export.
• Setting the headers, for the visualization table/csv export
• Handling the data chunking and pagination for display and export.
• Preparing each line of data for display and export.

A singular Audience Definition then need to define:

• getQuery(): The base query to retrieve the relevant data from the database.
• fields(): The transformations and the structure of the data to be exported or displayed.

Data Chunking and Pagination

To optimize memory usage and improve performance when handling large datasets:

• Displaying Data on Page: Data is fetched and displayed in chunks of 1000 records per page. Pagination controls are provided to navigate through different chunks.
• Downloading CSVs: When exporting data, the system processes data in chunks of 1000 records and returns a streamed response. This approach prevents memory overload by not loading the entire dataset into memory at once.

Testing

Testing can be straightforward on some Audience lists and more complex on others. If you try using NewlyJoinedIndividualMembers you'll see the module functionality working locally without much effort.

However, if you try using AllNewsletterSubscribers or AllAlmostCompanyMembers you might run into some issues.

There are some common problems you can find when testing Audience lists.

Missing production tables

Some Audience lists are related to data that is not available on the local database. Please check if the data your exporting is included in your local database, and if not you can follow the guide here to import missing tables.

Memory issues

Dev dependencies like Ray and Ignition can have an impact on memory consumption, especially when handling big datasets that query the database often. This happens because all the queries are stored in memory by these packages. If you are experiencing memory issues, you can run composer install --no-dev to remove these dependencies resulting in an environment closer to production.