Test Streams with Docker Compose

4 minute read

Prerequisites

• Docker 19.03.02+
• Docker-compose 1.17.1+
• Resources: at least 8 CPUs and 12 GB RAM

Prepare your environment

After you have been onboarded on Amplify Platform, you will be able to download our latest Docker Compose archive from the Downloads section of the Axway Support portal. Ensure to download the correct version of the Streams docker compose archive corresponding to the release you wish to test.

To prepare your environment, extract the Docker Compose archive and open a terminal from the extracted directory.

Use Amplify Platform as your Docker registry

Docker images must be hosted in a docker registry accessible from your local computer. We recommend you to use the Amplify Platform repository for a custom docker registry. Alternatively, you can use your own custom Docker registry.

To use the Amplify Platform as your container registry you must first ensure the following:

• You can see our images with your organization on the Amplify repository search page.
• You have administrator access to create a service account in your organization.

After you have verified that your images are loaded and that you have the correct level of access, you must create a service account, then create docker-registry secret with the information from your service account.

Create a service account

To create your service account, perform the following steps:

1. Log in to the Amplify Platform.
2. Select your organization, and from the left menu, click Service Accounts (You should see all service accounts already created).
3. Click + Service Account, and fill in the mandatory fields:
   • Enter a name for the service account.
   • Choose Client Secret for the method.
   • Choose Platform-generated secret for the credentials.
4. Click Save
5. Ensure to securely store the generated client secret because it will be required in further steps.

Log in to your Axway repository

After creating your service account, run the following command to log in to your Axway repository:

export AXWAY_CLIENT_ID="<Client ID>"
export AXWAY_CLIENT_SECRET="<Client Secret>"

docker login docker.repository.axway.com -u ${AXWAY_CLIENT_ID} -p ${AXWAY_CLIENT_SECRET}

General conditions for license and subscription services

To proceed with the installation, you must accept Axway General Terms and Conditions:

“You hereby accept that the Axway Products and/or Services shall be governed exclusively by the Axway General Terms and Conditions, unless an agreement has been signed with Axway in which case such agreement shall apply.”

To accept the conditions you must add the STREAMS_ACCEPT_GENERAL_CONDITIONS=yes variable to your .env file.

Execute Docker Compose to start Streams

To start Streams, run the following command:

docker-compose up -d

Ensure all the services are up and running

You can run the process status command to check the status of the containers:

docker-compose ps

Result:

Name                            Command                          State          Ports                                         
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
streams-database                /opt/bitnami/scripts/maria ...   Up             0.0.0.0:3306->3306/tcp,:::3306->3306/tcp                                              
streams-hub                     bash scripts/wait-for-it.s ...   Up (healthy)   0.0.0.0:50001->50001/tcp,:::50001->50001/tcp, 0.0.0.0:9001->8080/tcp,:::9001->8080/tcp
streams-kafka                   /opt/bitnami/scripts/kafka ...   Up             0.0.0.0:29092->29092/tcp,:::29092->29092/tcp, 0.0.0.0:9092->9092/tcp,:::9092->9092/tcp
streams-prometheus              /opt/bitnami/prometheus/bi ...   Up             0.0.0.0:9090->9090/tcp,:::9090->9090/tcp                                              
streams-publisher-http-poller   bash scripts/entrypoint.sh       Up (healthy)   0.0.0.0:50002->50002/tcp,:::50002->50002/tcp, 0.0.0.0:9002->8080/tcp,:::9002->8080/tcp
streams-publisher-http-post     bash scripts/entrypoint.sh       Up (healthy)   0.0.0.0:50004->50004/tcp,:::50004->50004/tcp, 0.0.0.0:9004->8080/tcp,:::9004->8080/tcp
streams-publisher-sfdc          bash scripts/entrypoint.sh       Up (healthy)   0.0.0.0:50006->50006/tcp,:::50006->50006/tcp, 0.0.0.0:9006->8080/tcp,:::9006->8080/tcp
streams-subscriber-sse          bash scripts/entrypoint.sh       Up (healthy)   0.0.0.0:50000->50000/tcp,:::50000->50000/tcp, 0.0.0.0:9000->8080/tcp,:::9000->8080/tcp
streams-subscriber-webhook      bash scripts/wait-for-it.s ...   Up (healthy)   0.0.0.0:50003->50003/tcp,:::50003->50003/tcp, 0.0.0.0:9003->8080/tcp,:::9003->8080/tcp
streams-zookeeper               /opt/bitnami/scripts/zooke ...   Up             0.0.0.0:2181->2181/tcp,:::2181->2181/tcp, 2888/tcp, 3888/tcp, 8080/tcp    

Run smoke tests to verify Streams configuration

To verify Streams is properly configured, launch the built-in smoke tests. The tests perform the following:

• Test the Hub API by creating a topic named smoke-test-topic using our public test API https://stockmarket.streamdata.io/prices and the http-poller publisher.
• Test the SSE subscription API by starting a subscription to the topic previously created.
• If the test is successful, the container exits with code 0, otherwise an error is displayed.

Run the following command to start the smoke tests:

docker-compose -f docker-compose.test.yml up

The following shows the output in case of success:

sut_1  | PUBLISHER_HTTP_POLLER_URL=http://streams-publisher-http-poller:8080
sut_1  | HUB_URL=http://streams-hub:8080
sut_1  | SUBSCRIBER_SSE_URL=http://streams-subscriber-sse:8080
sut_1  | TARGET_URL=http://stockmarket.streamdata.io/prices
sut_1  | TOPIC_CONFIG={ "name": "smoke-test-topic-5496", "publisher": {"type": "http-poller","config": {"url": "http://stockmarket.streamdata.io/prices","pollingPeriod": "PT1S"}}}
sut_1  |
sut_1  | Publisher.HTTP-Poller: UP
sut_1  |                   Hub: UP
sut_1  |        Subscriber.SSE: UP
sut_1  | 
sut_1  | Testing Hub topic creation API...
sut_1  | Successful topic creation: 1f5fe57c-0b83-415e-81c1-54707458b3c3
sut_1  | Testing SSE subscription to topic: 1f5fe57c-0b83-415e-81c1-54707458b3c3
sut_1  | Successful subscription to topic 1f5fe57c-0b83-415e-81c1-54707458b3c3 events 'snapshot,patch,patch,patch,patch,patch,patch,'
streams_sut_1 exited with code 0

Stop Streams

To stop all services, execute the following command docker-compose down.