This directory contains utility scripts for managing and maintaining tests in the RudderStack transformer project.

The migrateTest.ts script helps migrate legacy test files to a new, optimized format.

ts-node test/scripts/migrateTest.ts -d DESTINATION_NAME -f FEATURE_TYPE

Options:

• -d, --destination <string>: Destination name to migrate (required)
• -f, --feature <type>: Feature type (processor/router/proxy), defaults to "processor"

This document outlines a comprehensive strategy for migrating RudderStack transformer tests to the new optimized format using the migrateTest.ts utility. The migration process aims to standardize test structure, reduce duplication, and improve maintainability.

Migrating tests to the new format provides several key benefits:

1. Improved Type Safety: The new format uses TypeScript interfaces for better type checking and code completion.

2. Enhanced Documentation: The new format encourages documenting test cases with descriptions, scenarios, and success criteria.

3. Better Test Visualization: Migrated tests can be visualized using Allure reports, making it easier to understand test coverage and results.

4. Standardized Structure: All tests follow a consistent structure, making them easier to maintain and extend.

5. Reduced Duplication: Common values like metadata and destination configurations are extracted to reduce repetition.

6. Zod Schema Validation: Tests can be validated using Zod schemas to ensure they conform to expected formats.

Based on the examination of migrated tests, there are two main patterns for organizing test files:

This pattern is used for simpler destinations and has the following characteristics:

• A single data.ts file containing all test cases
• Common values (metadata, destination config) extracted to variables at the top of the file
• Test cases reference these common values
• Example: active_campaign destination

For destinations with multiple feature types (processor, router), a variation of Pattern 1 can be used:

• A separate common.ts file containing shared code and configurations
• Feature-specific data.ts files (e.g., in processor/ and router/ directories) that import from common.ts
• Helper functions for generating common structures (messages, metadata, etc.)
• Leveraging existing utility functions from testUtils.ts (like generateMetadata, overrideDestination)
• Example: slack destination

This pattern is used for more complex destinations with many test cases:

• Main data.ts file that imports and combines test cases from multiple files
• Separate files for different event types (identify, track, screen, etc.)
• Each file contains test cases specific to that event type
• Common values shared across files
• Example: klaviyo destination

The migration script supports both patterns, with Pattern 1 being the default for simpler destinations.

Migrated tests follow a standardized structure with enhanced metadata. Depending on the feature type (processor, router, etc.), different TypeScript interfaces should be used:

For simple test cases or when specific types aren't applicable:

export const testCases: TestCaseData[] = [
  {
    id: 'unique-test-id',
    name: 'destination_name',
    module: 'destination',
    version: 'v0',
    feature: 'processor',
    description: 'Detailed description of what this test verifies',
    scenario: 'Business scenario being tested',
    successCriteria: 'What defines a successful test',
    tags: ['tag1', 'tag2'],
    input: {
      request: {
        method: 'POST',
        headers: {
          /* headers */
        },
        body: {
          /* request body */
        },
      },
    },
    output: {
      response: {
        status: 200,
        body: {
          /* expected response */
        },
      },
    },
    mockFns: (mockAdapter) => {
      // Mock HTTP requests
    },
  },
];

For processor tests, use the ProcessorTestData type:

import { ProcessorTestData } from '../../../../integrations/testTypes';

export const data: ProcessorTestData[] = [
  {
    id: 'destination-processor-test',
    name: 'destination_name',
    description: 'Test description',
    scenario: 'Business scenario',
    successCriteria: 'Success criteria',
    feature: 'processor',
    module: 'destination',
    version: 'v0',
    input: {
      request: {
        method: 'POST',
        body: [
          {
            message: {
              /* RudderMessage */
            },
            metadata: {
              /* Metadata */
            },
            destination: {
              /* Destination */
            },
          },
        ],
      },
    },
    output: {
      response: {
        status: 200,
        body: [
          {
            output: {
              /* ProcessorTransformationOutput */
              version: '1',
              type: 'REST',
              method: 'POST',
              endpoint: 'https://api.example.com',
              headers: {
                /* headers */
              },
              params: {
                /* params */
              },
              body: {
                JSON: {
                  /* JSON body */
                },
                XML: {},
                JSON_ARRAY: {},
                FORM: {},
              },
              files: {},
              userId: 'user123',
            },
            metadata: {
              /* Metadata */
            },
            statusCode: 200,
          },
        ],
      },
    },
    mockFns: (mockAdapter) => {
      // Mock HTTP requests
    },
  },
];

When using ProcessorTestData and RouterTestData types, be aware that there's a discrepancy between the type definitions and the actual expected structure:

• The ProcessorTransformationOutput type (used in both processor and router tests) doesn't include a statusCode property according to the type definition.
• However, many existing tests include a statusCode property in the batchedRequest objects.
• This can cause TypeScript errors when using the specific types.

To avoid these errors, you can:

1. Use the generic TestCaseData type instead of the specific types
2. Remove the statusCode property from the batchedRequest objects
3. Cast the objects to any or use type assertions

For router tests, use the RouterTestData type:

import { RouterTestData } from '../../../../integrations/testTypes';

export const data: RouterTestData[] = [
  {
    id: 'destination-router-test',
    name: 'destination_name',
    description: 'Test description',
    scenario: 'Business scenario',
    successCriteria: 'Success criteria',
    feature: 'router',
    module: 'destination',
    version: 'v0',
    input: {
      request: {
        method: 'POST',
        body: {
          input: [
            {
              message: {
                /* RudderMessage */
              },
              metadata: {
                /* Metadata */
              },
              destination: {
                /* Destination */
              },
            },
          ],
          destType: 'destination_name',
        },
      },
    },
    output: {
      response: {
        status: 200,
        body: {
          output: [
            {
              batched: false,
              batchedRequest: [
                {
                  /* ProcessorTransformationOutput */
                  version: '1',
                  type: 'REST',
                  method: 'POST',
                  endpoint: 'https://api.example.com',
                  headers: {
                    /* headers */
                  },
                  params: {
                    /* params */
                  },
                  body: {
                    JSON: {
                      /* JSON body */
                    },
                    XML: {},
                    JSON_ARRAY: {},
                    FORM: {},
                  },
                  files: {},
                  userId: 'user123',
                },
              ],
              destination: {
                /* Destination */
              },
              metadata: [
                /* Metadata */
              ],
              statusCode: 200,
            },
          ],
        },
      },
    },
    mockFns: (mockAdapter) => {
      // Mock HTTP requests
    },
  },
];

• id: Unique identifier for the test case
• name: Destination name (lowercase)
• module: Always "destination" for destination tests (lowercase)
• version: API version (usually "v0")
• feature: Test type (processor, router, dataDelivery)
• description: Detailed description of what the test verifies
• scenario: Business scenario being tested
• successCriteria: What defines a successful test
• tags: Optional tags for categorizing tests
• input: Request data
• output: Expected response
• mockFns: Function to mock HTTP requests

The testUtils.ts file provides several utility functions that should be used in test files to maintain consistency and reduce duplication:

Use this function to override specific properties in a destination configuration:

import { overrideDestination } from '../../../testUtils';

const destination = overrideDestination(baseDestination, {
  configKey1: 'value1',
  configKey2: 'value2',
});

Use this function to generate metadata objects with consistent structure:

import { generateMetadata } from '../../../testUtils';

const metadata = generateMetadata(jobId, userId, messageId);

Use this function to create output objects with the correct structure:

import { transformResultBuilder } from '../../../testUtils';

const output = transformResultBuilder({
  version: '1',
  type: 'REST',
  method: 'POST',
  endpoint: 'https://api.example.com',
  headers: {
    /* headers */
  },
  params: {
    /* params */
  },
  JSON: {
    /* JSON body */
  },
  userId: 'user123',
});

Use these functions to generate message objects with consistent structure:

import {
  generateTrackPayload,
  generateIndentifyPayload,
  generatePageOrScreenPayload,
  generateGroupPayload,
} from '../../../testUtils';

const trackMessage = generateTrackPayload({
  event: 'event_name',
  properties: {
    /* properties */
  },
  context: {
    /* context */
  },
});

const identifyMessage = generateIndentifyPayload({
  traits: {
    /* traits */
  },
  context: {
    /* context */
  },
});

Migrated tests are automatically validated using Zod schemas when the destination is added to the INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array in test/integrations/component.test.ts. This array currently includes:

const INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE = [
  'active_campaign',
  'klaviyo',
  'campaign_manager',
  'criteo_audience',
  'branch',
  'userpilot',
  'loops',
  'slack',
  'braze',
];

When a destination is added to this array, the component tests will:

1. Validate the test data against Zod schemas
2. Generate Allure reports with detailed test information
3. Provide enhanced error reporting with JSON diffs

• Inventory existing tests: Create a list of all destinations and their test files
• Prioritize destinations: Identify high-priority destinations based on usage and complexity
• Create a migration schedule: Plan the migration in batches to minimize disruption

• Set up a dedicated branch: Create a feature branch for the migration work
• Ensure dependencies: Verify all required dependencies are installed
• Prepare rollback plan: Establish a process for reverting changes if issues arise

• Start with simple destinations: Begin with less complex destinations to validate the process
• Migrate by feature type: Process one feature type at a time (processor, router, proxy)
• Command template:

  ts-node test/scripts/migrateTest.ts -d DESTINATION_NAME -f FEATURE_TYPE

For each migrated destination:

• Run tests: Execute the migrated tests to ensure functionality is preserved
• Compare output: Verify the test results match pre-migration behavior
• Review optimizations: Check that common values were properly extracted

The test migration is part of a broader initiative to improve test visualization and documentation. Migrated tests can be visualized using Allure, a flexible lightweight test report tool.

• Visual Test Reports: Generates comprehensive HTML reports with detailed test information
• Test Case Organization: Groups tests by features, epics, and stories
• Detailed Test Information: Shows test descriptions, scenarios, and success criteria
• Request/Response Visualization: Displays input and output data for each test
• Failure Analysis: Provides detailed diffs for failed tests

1. When tests are migrated to the new format, they include additional metadata like:

   • description: Detailed description of what the test verifies
   • scenario: The business scenario being tested
   • successCriteria: What defines a successful test
   • tags: Optional tags for categorizing tests

2. The Allure reporter uses this metadata to generate comprehensive reports:

   • Tests are organized by destination, feature, and scenario
   • Test details include request/response data
   • Failed tests show detailed diffs between expected and actual results

3. The integration is implemented in test/test_reporter/allureReporter.ts with these key components:

   • enhancedTestReport: Generates detailed reports with JSON diffs for failed tests
   • enhancedTestUtils: Provides utilities for test setup, execution, and reporting

4. For each test case, the Allure reporter:

   • Creates a hierarchical structure (epic → feature → story)
   • Attaches request and response data as JSON
   • Generates detailed diffs for failed tests
   • Provides visual indicators of test status

5. The component test framework automatically integrates with Allure when a destination is added to the INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array:

// In component.test.ts
if (INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE.includes(tcData.name?.toLocaleLowerCase())) {
  expect(validateTestWithZOD(tcData, response)).toEqual(true);
  enhancedTestUtils.beforeTestRun(tcData);
  enhancedTestUtils.afterTestRun(tcData, response.body);
}

The migration script adds placeholders for mock functions. You'll need to manually implement these functions:

// Before migration
{
  // Test case data
  mockFns: (mockAdapter) => {
    mockAdapter.onPost('https://api.example.com').reply(200, { success: true });
  };
}

// After migration
{
  // Test case data
  mockFns: 'Add mock of index 0';
}

To fix this, you need to:

1. Identify test cases with mock functions in the original files (check the backup files)
2. Implement the mock functions in the migrated files
3. Replace the placeholder string with the actual implementation

Example of a properly implemented mock function after migration:

mockFns: (mockAdapter: MockAdapter) => {
  mockAdapter
    .onPost(
      'https://api.example.com',
      {
        asymmetricMatch: (actual) => {
          return isMatch(actual, {
            // Expected request body pattern
          });
        },
      },
      {
        asymmetricMatch: (actual) => {
          return isMatch(actual, {
            // Expected headers pattern
          });
        },
      },
    )
    .reply(200, { success: true });
};

For destinations listed in INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array in test/integrations/component.test.ts, additional validation is performed using Zod schemas. After migration, you should add your destination to this array if it's using the new test structure.

Zod schema validation provides several benefits:

• Runtime Type Checking: Validates that test data conforms to expected schemas
• Improved Error Messages: Provides detailed error messages when validation fails
• Consistent Data Structure: Ensures all tests follow the same structure

The validation process works as follows:

1. When a test runs, the input and output data are validated against predefined Zod schemas
2. These schemas are defined in src/types/zodTypes.ts and include:
   • ProcessorTransformationResponseListSchema: For processor test responses
   • RouterTransformationResponseListSchema: For router test responses
   • DeliveryV0ResponseSchema and DeliveryV1ResponseSchema: For proxy test responses
3. If validation fails, the test will fail with a detailed error message

To enable Zod validation for your migrated tests:

1. Add your destination to the INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array
2. Ensure your test data conforms to the expected schemas
3. Run the tests to verify validation passes

• Address complex cases: Manually adjust any tests that weren't properly migrated
• Update documentation: Update any documentation referencing the test structure

• Comprehensive testing: Run the full test suite to ensure all tests pass
• Code review: Have team members review the migrated tests
• Performance check: Verify that test execution time hasn't significantly increased

• Remove backup files: Once verified, remove the .backup.ts files
• Update references: Update any external references to the test files
• Commit changes: Finalize changes with descriptive commit messages

• Evaluate test complexity and choose appropriate pattern:
  • Pattern 1 (Single file) for simpler destinations
  • Pattern 2 (Multiple files) for complex destinations with many test cases
• Migrate processor tests
• Verify processor tests functionality
• Migrate router tests
• Verify router tests functionality
• Migrate proxy tests (N/A - Slack doesn't have proxy tests)
• Verify proxy tests functionality (N/A - Slack doesn't have proxy tests)
• Fix any mock functions
• Add destination to INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array in component.test.ts
• Remove backup files after verification
• Further improvements (May 2024):
  • Create dedicated common.ts file for shared code
  • Extract common destination configurations
  • Create helper functions for common structures
  • Leverage existing utility functions from testUtils.ts
  • Remove unnecessary fields from metadata
  • Verify all tests pass with new structure

• Evaluate test complexity and choose appropriate pattern:
  • Pattern 1 (Single file) for simpler destinations
  • Pattern 2 (Multiple files) for complex destinations with many test cases
• Migrate processor tests (N/A - Not part of this migration)
• Verify processor tests functionality (N/A - Not part of this migration)
• Migrate router tests (N/A - Not part of this migration)
• Verify router tests functionality (N/A - Not part of this migration)
• Migrate dataDelivery tests (equivalent to proxy)
• Verify dataDelivery tests structure
• Fix any mock functions
• Add destination to INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array in component.test.ts
• Remove backup files after verification

For each destination:

• Evaluate test complexity and choose appropriate pattern:
  • Pattern 1 (Single file) for simpler destinations
  • Pattern 2 (Multiple files) for complex destinations with many test cases
• Migrate processor tests
• Verify processor tests functionality
• Migrate router tests
• Verify router tests functionality
• Migrate proxy/dataDelivery tests
• Verify proxy/dataDelivery tests functionality
• Fix any mock functions
• Add destination to INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array in component.test.ts
• Remove backup files after verification

1. Migrate one destination at a time: Complete the full migration cycle for each destination before moving to the next
2. Commit frequently: Make small, focused commits for each migration step
3. Document issues: Keep track of any issues encountered during migration for future reference
4. Maintain test coverage: Ensure test coverage doesn't decrease during migration
5. Validate with CI/CD: Use continuous integration to validate migrated tests

• Solution: Compare with backup files to identify differences
• Prevention: Run tests immediately after migration

• Solution: Manually implement mock functions based on original implementation
• Prevention: Document complex mock functions before migration

• Solution: Manually adjust the extracted common values
• Prevention: Review test cases for unusual patterns before migration

The Slack destination was successfully migrated to the new test format as part of the ongoing test modernization effort. This case study documents the process, challenges, and solutions encountered during the migration.

1. Initial Assessment:

   • Slack destination had both processor and router tests
   • Tests were relatively simple with a small number of test cases
   • Pattern 1 (Single file with extracted common values) was chosen for the migration

2. Migration Steps:

   • Processor tests were migrated first using the migration script:

     npx ts-node test/scripts/migrateTest.ts -d slack -f processor

   • Router tests were migrated next:

     npx ts-node test/scripts/migrateTest.ts -d slack -f router

   • The destination was added to the INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array

3. Challenges and Solutions:

   • Challenge: The migration script didn't properly update the router tests to the new format
   • Solution: Manually updated the router tests to match the new format
   • Challenge: The migrated tests had TypeScript errors related to the statusCode property
   • Solution: Added the statusCode field to the output objects to match the actual response structure

4. Verification:

   • All tests were run to verify functionality:

     npm run test:ts -- component --destination=slack

   • Tests passed successfully, confirming the migration was successful

The Slack destination migration resulted in:

• Improved test structure with better documentation
• Extracted common values to reduce duplication
• Added proper type safety with TypeScript interfaces
• Enhanced test reporting through Allure integration

The Slack destination tests were further improved by:

• Creating a dedicated common.ts file to share code between processor and router tests
• Extracting common destination configurations into the shared file
• Creating helper functions for generating common message structures and metadata
• Leveraging existing utility functions from testUtils.ts like generateMetadata and overrideDestination to avoid code duplication
• Removing unnecessary fields like anonymousId from metadata to align with schema requirements
• Ensuring all tests pass with the new structure

1. Manual Intervention: While the migration script automates much of the process, manual intervention is often needed for specific edge cases
2. Test Structure Understanding: Understanding the expected output structure is crucial for successful migration
3. Incremental Approach: Migrating one feature type at a time (processor, then router) helps isolate and resolve issues

The Braze destination was migrated to the new test format, focusing on the dataDelivery feature. This case study highlights the challenges encountered with mock functions and feature type handling.

1. Initial Assessment:

   • Braze destination had dataDelivery tests with mock functions
   • Tests used axios-mock-adapter for mocking HTTP requests
   • Pattern 1 (Single file with extracted common values) was chosen for the migration

2. Migration Steps:

   • Updated the migration script to handle dataDelivery feature type:

     // Normalize feature type - treat dataDelivery as proxy
     const normalizedFeature =
       feature.toLowerCase() === 'datadelivery' ? 'proxy' : feature.toLowerCase();

   • Ran the migration script for the dataDelivery feature:

     npx ts-node test/scripts/migrateTest.ts -d braze -f dataDelivery

   • Added the braze destination to the INTEGRATIONS_WITH_UPDATED_TEST_STRUCTURE array

3. Challenges and Solutions:

   • Challenge: The migration script didn't recognize the dataDelivery feature type
   • Solution: Updated the script to treat dataDelivery as an alias for proxy
   • Challenge: Mock functions were not properly migrated
   • Solution: Manually added the mock functions back to the migrated tests
   • Challenge: Type errors in the migrated file
   • Solution: Fixed type issues by removing explicit typing that was causing conflicts

4. Verification:

   • Ran the tests to verify functionality:

     npm run test:ts -- component --destination=braze

   • Tests failed due to missing mock implementations, but the structure was correct

The Braze destination migration resulted in:

• Improved script to handle dataDelivery feature type
• Better understanding of how mock functions should be handled
• Identification of type issues that need to be addressed in the migration script

1. Feature Type Aliases: The migration script needs to handle different feature type names that are functionally equivalent
2. Mock Function Preservation: Special attention is needed to preserve mock functions during migration
3. Type Safety: The migration script should generate type-safe code that doesn't cause TypeScript errors

Following this structured approach will ensure a smooth migration of test files to the new format while maintaining test integrity and functionality. The migration process not only standardizes the test structure but also optimizes test files by reducing duplication, making them more maintainable in the long run.

The Slack and Braze destination migrations serve as practical examples of how to successfully apply this migration strategy to real-world test cases. They highlight different challenges that may be encountered during the migration process and provide solutions that can be applied to future migrations.