Efficient SQS Messaging in Spring Boot with SqsPump: A Complete Guide

The component provides a highly efficient way to send messages to AWS SQS queues in Spring Boot applications through intelligent batching and optimized throughput. This article explores how to integrate and leverage this powerful messaging utility in your Spring Boot applications. 

Our open source library, sqs-utilities, is available on GitHub here: sqs-utilities

Adding SQS Utilities to Your Maven Dependencies

To include the library in your Maven project, you need to add the appropriate dependency to your file. Based on the project structure, here’s how to configure it:

Maven Dependency Configuration

Add the following dependency to your file’s <dependencies> section: pom.xml

<dependencies>
<!-- SQS Utilities for AWS SQS messaging -->
<dependency>
<groupId>com.limemojito.oss.standards.aws</groupId>
<artifactId>sqs-utilities</artifactId>
<version>15.3.3</version>
</dependency>
</dependencies>
<dependencies>
    <!-- SQS Utilities for AWS SQS messaging -->
    <dependency>
        <groupId>com.limemojito.oss.standards.aws</groupId>
        <artifactId>sqs-utilities</artifactId>
        <version>15.3.3</version>
    </dependency>
</dependencies>

Getting Started with SqsPumpConfig

To enable SqsPump in your Spring Boot application, simply import the configuration class:

@SpringBootApplication
@Import(SqsPumpConfig.class)
public class MyApplication {
    public static void main(String[] args) {
        SpringApplication.run(MyApplication.class, args);
    }
}

The @Import(SqsPumpConfig.class) annotation automatically configures all necessary components:

  • The main batching component SqsPump
  • The underlying AWS SQS client wrapper SqsSender
  • JSON serialization with Spring Boot-like configuration ObjectMapper
  • AWS SDK v2 client (must be provided as a bean) SqsClient

Basic Usage in Your Components

Once configured, inject into any Spring component: SqsPump

@Service
public class OrderProcessingService {
    
    private final SqsPump sqsPump;
    private static final String ORDER_QUEUE_URL = "https://sqs.region.amazonaws.com/account/order-queue";
    
    public OrderProcessingService(SqsPump sqsPump) {
        this.sqsPump = sqsPump;
    }
    
    public void processOrder(Order order) {
        // Add order to batch - efficient, non-blocking
        sqsPump.send(ORDER_QUEUE_URL, order);
        
        // Batch will be sent automatically when full or via explicit flush
    }
    
    public void forceFlushPendingOrders() {
        sqsPump.flush(ORDER_QUEUE_URL);
    }
}

Understanding Batch Efficiency

The core efficiency of comes from its intelligent batching mechanism powered by the SqsSender.sendBatch() method. Here’s how it optimizes throughput: SqsPump

Note that you should flush(destination) on SqsPump at the end of your batch processing to clear the batch queue.

Automatic Batching Strategy

// Configuration
com.limemojito.sqs.batchSize=10 # Default batch size

The pump accumulates messages until:

  1. Batch size reached: When 10 messages (default) are queued
  2. Explicit flush: When you call flush(destination)
  3. Application shutdown: Via lifecycle hook @PreDestroy

Efficiency Calculations

For high-volume scenarios, the efficiency gains are significant:

Without Batching (Individual sends):

  • 1000 messages = 1000 API calls
  • Each call: ~50-100ms latency
  • Total time: 50-100 seconds

With SqsPump Batching:

  • 1000 messages = 100 batch calls (10 messages each)
  • Each batch call: ~50-100ms latency
  • Total time: 5-10 seconds
  • 90% reduction in execution time

Cost Optimization

AWS SQS pricing is per request, making batching extremely cost-effective:

  • Individual sends: 1000 requests × 0.0000004 =0.0004
  • Batch sends: 100 requests × 0.0000004 =0.00004
  • 90% cost reduction

Advanced Usage Patterns

Multi-Queue Processing

@Service
public class EventPublisher {
    
    private final SqsPump sqsPump;
    
    public void publishEvents(List<Event> events) {
        for (Event event : events) {
            String queueUrl = determineQueueForEvent(event);
            sqsPump.send(queueUrl, event);
        }
        
        // Flush all queues at once for maximum efficiency when using multiple destinations. Use flush(queueUrl) for single destination.
        sqsPump.flushAll();
    }
}

FIFO Queue Support

For ordered message processing:

@Service
public class SequentialProcessor {
    
    private final SqsPump sqsPump;
    private static final String FIFO_QUEUE = "https://sqs.region.amazonaws.com/account/ordered-queue.fifo";
    
    public void sendOrderedMessage(String groupId, List<Object> messages, String deduplicationId) {
        Map<String, Object> fifoHeaders = Map.of(
            "message-group-id", groupId,
            "message-deduplication-id", deduplicationId
        );
        messages.forEach(message -> sqsPump.send(FIFO_QUEUE, message, fifoHeaders));
        // send any remaining messages
        sqsPump.flush(FIFO_QUEUE);
    }
}

Thread Safety and Concurrency

is designed for high-concurrency environments:

Key Thread Safety Features:

  • SqsPump: Thread-safe message storage per destination ConcurrentHashMap
  • :SqsPump: Lock-free message queuing ConcurrentLinkedDeque
  • Synchronized flush: Only one thread flushes per destination at a time, no explicit coding necessary.
  • Atomic batch operations: Complete batches or failure with rollback

Configuration Options

Application Properties

# application.yml
com:
limemojito:
sqs:
batchSize: 10 # Max messages per batch (default: 10, max: 10 per AWS limits)

# AWS SQS Client configuration
aws:
region: us-west-2
credentials:
accessKey: ${AWS_ACCESS_KEY}
secretKey: ${AWS_SECRET_KEY}

Custom SQS Client Configuration

@Configuration
public class AwsConfig {

@Bean
public SqsClient sqsClient() {
return SqsClient.builder()
.region(Region.US_WEST_2)
.credentialsProvider(DefaultCredentialsProvider.create())
.build();
}
}

Message Attributes and Spring Compatibility

SqsPump automatically adds Spring Messaging-compatible attributes: 

{
"MessageAttributes": {
"id": "uuid-generated",
"timestamp": "1640995200000",
"contentType": "application/json",
"Content-Type": "application/json",
"Content-Length": "156"
}
}

These attributes ensure seamless integration with:

  • Spring Cloud Stream
  • Spring Integration
  • Spring Boot messaging auto-configuration

Best Practices

1. Flush when needed

After looping, always flush. Messages may be in memory before delivery.

messages.forEach(message -> sqsPump.send(FIFO_QUEUE, message, fifoHeaders));
// send any remaining messages
sqsPump.flush(FIFO_QUEUE);

2. Use Explicit Flushing for Critical Messages

// For time-sensitive messages
sqsPump.send(queueUrl, criticalMessage);
sqsPump.flush(queueUrl); // Immediate send

3. Batch Size Optimization

// For high-throughput: use max batch size
com.limemojito.sqs.batchSize=10

// For low-latency: use smaller batches
com.limemojito.sqs.batchSize=3

4. Lifecycle Management

SqsPump automatically flushes pending messages on container shutdown in @PreDestroy.

Conclusion

transforms SQS messaging in Spring Boot applications by providing:

  • 90% reduction in API calls through intelligent batching
  • Significant cost savings via reduced request counts
  • Thread-safe concurrent message handling
  • Spring Boot integration with zero configuration overhead
  • Automatic lifecycle management preventing message loss

By leveraging SqsSender.sendBatch() under the hood, delivers enterprise-grade performance while maintaining the simplicity that Spring Boot developers expect. Whether you’re processing thousands of messages per second or need reliable ordered delivery via FIFO queues, provides the foundation for scalable, efficient AWS SQS integration. SqsPumpSqsPump

The combination of automatic batching, thread safety, and Spring Boot’s dependency injection makes an ideal choice for modern cloud-native applications requiring high-performance message queuing.

For information on the testing of SQSPump, see our article here.

Testing High Performance AWS SQS batch sending

The class, from our OSS Maven standards project, is a well-structured unit test suite that demonstrates testing patterns for AWS SQS batch messaging functionality. As a senior Java developer, you’ll appreciate the sophisticated test design patterns and AWS integration testing techniques employed in this test class.

See the test code here on GitHub: SqsPumpTest.java

For more information on our open source SQSPump, see our article here.

Test Architecture Overview

The test class follows modern Java testing best practices using JUnit 5, Mockito, and AssertJ. It’s designed to test the component, which is a utility for efficiently sending multiple messages to AWS SQS queues in batches using SqsPump. The code for SqsPump is available GitHub: SqsPump.java

Key Testing Components

Test Setup:

  • Uses for clean dependency injection @ExtendWith(MockitoExtension.class)
  • Mocks the to isolate the unit under test SqsClient
  • Mockito verify the exact parameters passed to AWS SDK calls ArgumentCaptor
  • Configures a controlled batch size of 10 messages for predictable testing

Core Dependencies:

  • The main class being tested SqsPump
  • A wrapped AWS SQS client with JSON serialization capabilities SqsSender
  • Jackson : Configured with boot-like settings for JSON processing ObjectMapper

Test Scenarios Explained

1. Batch Size Boundary Testing

The test suite includes three critical batch tests that demonstrate the pump’s ability to handle different message volumes:

  • Exact batch size (10 messages): Verifies optimal batching behavior
  • Large volume (100 messages): Tests multiple batch processing
  • Irregular size (33 messages): Ensures proper handling of partial final batches

These tests use the performBatchTest() method, which:

  1. Sends N objects to the pump TestMessage
  2. Triggers a flush operation
  3. Verifies the exact number of batch calls to SQS
  4. Validates that messages are properly batched and serialized

2. FIFO Queue Support

The test demonstrates support for FIFO (First-In-First-Out) queues, which require: shouldSendAFifoMessage()

  • Message Deduplication ID: Prevents duplicate processing
  • Message Group ID: Ensures ordering within message groups

This test verifies that FIFO-specific headers are correctly mapped to AWS SQS batch request parameters.

3. Edge Case Handling

The test ensures the pump optimizes for empty batches by not making unnecessary AWS API calls – a crucial performance consideration. shouldNotPumpZeroMessages()

Testing Patterns Worth Noting

1. Argument Capturing Pattern

@Captor
private ArgumentCaptor<SendMessageBatchRequest> requestCaptor;

This pattern allows precise verification of complex objects passed to mocked dependencies without exposing internal implementation details.

2. Batch Calculation Logic

The expectedBatchSends() method demonstrates a clean mathematical approach to calculating expected batch counts:

private int expectedBatchSends(int sendSize) {
return sendSize / pumpMaxBatchSize + (sendSize % pumpMaxBatchSize > 0 ? 1 : 0);
}

3. Message Attribute Validation

The test includes comprehensive validation of SQS message attributes, ensuring proper content-type headers, timestamps, and content length metadata are set correctly.

AWS SDK Integration Insights

The test reveals several important aspects of AWS SQS batch processing:

  • Batch Request Structure: Each batch contains multiple objects SendMessageBatchRequestEntry
  • Message Serialization: JSON serialization with proper content-type headers
  • Attribute Metadata: Automatic inclusion of timestamps and content length
  • ID Management: Sequential ID assignment for batch entries

Key Testing Techniques for Senior Developers

  1. Mock Strategy: Only the AWS client is mocked, allowing real business logic to execute
  2. Data-Driven Testing: Multiple test cases with different batch sizes validate edge cases
  3. Comprehensive Assertions: Both structural (batch count) and content (message bodies) validation
  4. Performance Considerations: Zero-message optimization testing prevents unnecessary API calls

Conclusion

This test suite exemplifies professional-grade testing for cloud-native Java applications. It demonstrates how to effectively test complex batch processing logic while maintaining clear test isolation and comprehensive coverage. The patterns used here are directly applicable to testing other AWS service integrations and batch processing scenarios in enterprise Java applications.

The test validates FIFO queue support, message attributes, and batch boundary conditions showing the maturity expected in production AWS integrations, making this an excellent reference for similar testing challenges.