In distributed systems, ensuring that certain operations execute only once across multiple instances is a critical requirement. Whether you’re processing payments, sending notifications, or performing data migrations, you need guarantees that these operations don’t accidentally run multiple times. This article explores how to use PostgreSQL advisory locks through the lock-postgres
utility to create cluster-safe once-only methods in Java.
See here for our OSS implementation of cluster locks using postgresql. We also have an implementation using DynamoDB here (same API).
The Challenge of Distributed Execution
Consider a common scenario: you have multiple instances of your application running in a cluster, and each instance processes scheduled tasks. Without proper coordination, you might end up with:
- Duplicate payment processing
- Multiple notification emails sent
- Race conditions in data migrations
- Resource contention issues
Traditional Java synchronization mechanisms like synchronized
blocks or ReentrantLock
only work within a single JVM. For cluster-wide coordination, you need a distributed locking mechanism.
Solution 1: Using PostgreSQL Advisory Locks
PostgreSQL provides advisory locks – lightweight, application-level locks that don’t interfere with table-level locks. These locks are perfect for coordinating application logic across multiple instances.
The lock-postgres utility leverages PostgreSQL’s advisory lock functions:
pg_try_advisory_xact_lock(key)
– Non-blocking lock attemptpg_advisory_xact_lock(key)
– Blocking lock acquisition
These locks are automatically released when the database transaction commits or rolls back, making them ideal for transactional operations.
Solution 2: Using DynamoDB and the AWS AmazonDynamoDBLockClient
We also have a dynamo DB solution using the AWS AmazonDynamoDBLockClient as the implementation of the Lock API. This is an implementation of the same lock API in the examples in this article.
Setting Up the Dependencies
First, add the required dependencies to your project:
PostgreSQL implementation:
<dependency> <groupId>com.limemojito.oss.standards.lock</groupId> <artifactId>lock-postgres</artifactId> <version>15.3.2</version> </dependency>
DynamoDB Implementation
<dependency> <groupId>com.limemojito.oss.standards.lock</groupId> <artifactId>lock-dynamodb</artifactId> <version>15.3.2</version> </dependency>
Basic Usage Pattern
The PostgresLockService
implements the LockService
interface and provides two primary methods for lock acquisition:
@Service @RequiredArgsConstructor public class OnceOnlyService { private final LockService lockService; private final PaymentProcessor paymentProcessor; @Transactional public void processPaymentOnceOnly(String paymentId) { String lockName = "payment-processing-" + paymentId; // Try to acquire the lock - non-blocking Optional<DistributedLock> lock = lockService.tryAcquire(lockName); if (lock.isPresent()) { try (DistributedLock distributedLock = lock.get()) { // Only one instance will execute this block paymentProcessor.process(paymentId); log.info("Payment {} processed successfully", paymentId); } } else { log.info("Payment {} is already being processed by another instance", paymentId); } } }
Blocking vs Non-Blocking Lock Acquisition
The lock service provides two approaches:
1. Non-Blocking (tryAcquire
)
@Transactional public void tryProcessOnceOnly(String taskId) { Optional<DistributedLock> lock = lockService.tryAcquire("task-" + taskId); if (lock.isPresent()) { try (DistributedLock distributedLock = lock.get()) { // Process the task performCriticalOperation(taskId); } } else { // Task is being processed elsewhere, skip or handle accordingly log.info("Task {} is already being processed", taskId); } }
2. Blocking (acquire
)
@Transactional public void waitAndProcessOnceOnly(String taskId) { // This will wait until the lock becomes available try (DistributedLock lock = lockService.acquire("task-" + taskId)) { // Guaranteed to execute once the lock is acquired performCriticalOperation(taskId); } // Lock is automatically released when the transaction commits }
Real-World Example: Daily Report Generation
Let’s implement a practical example where multiple application instances need to coordinate daily report generation:
@Component @RequiredArgsConstructor @Slf4j public class DailyReportService { private final LockService lockService; private final ReportRepository reportRepository; private final NotificationService notificationService; @Scheduled(cron = "0 0 2 * * *") // Run at 2 AM daily @Transactional public void generateDailyReport() { String today = LocalDate.now().toString(); String lockName = "daily-report-" + today; Optional<DistributedLock> lock = lockService.tryAcquire(lockName); if (lock.isPresent()) { try (DistributedLock distributedLock = lock.get()) { log.info("Starting daily report generation for {}", today); // Check if report already exists (additional safety) if (reportRepository.existsByDate(today)) { log.info("Report for {} already exists, skipping", today); return; } // Generate the report Report report = generateReport(today); reportRepository.save(report); // Send notifications notificationService.sendReportGeneratedNotification(report); log.info("Daily report for {} generated successfully", today); } } else { log.info("Daily report for {} is being generated by another instance", today); } } private Report generateReport(String date) { // Implementation of report generation logic return new Report(date, collectDailyMetrics()); } }
Advanced Patterns
1. Lock with Timeout Handling
For blocking locks, you can implement timeout handling using Spring’s @Transactional
timeout:
@Transactional(timeout = 30) // 30-second timeout public void processWithTimeout(String taskId) { try (DistributedLock lock = lockService.acquire("timeout-task-" + taskId)) { performLongRunningOperation(taskId); } catch (DataAccessException e) { log.error("Failed to acquire lock within timeout period", e); throw new LockTimeoutException("Could not acquire lock for task: " + taskId); } }
2. Hierarchical Locking
Create hierarchical locks for complex operations:
@Transactional public void processOrderWithHierarchy(String customerId, String orderId) { // First acquire customer-level lock try (DistributedLock customerLock = lockService.acquire("customer-" + customerId)) { // Then acquire order-level lock try (DistributedLock orderLock = lockService.acquire("order-" + orderId)) { processOrderSafely(customerId, orderId); } } }
3. Conditional Processing with Fallback
@Transactional public ProcessingResult processWithFallback(String taskId) { Optional<DistributedLock> lock = lockService.tryAcquire("primary-task-" + taskId); if (lock.isPresent()) { try (DistributedLock distributedLock = lock.get()) { return performPrimaryProcessing(taskId); } } else { // Primary processing is happening elsewhere, perform alternative action return performAlternativeProcessing(taskId); } }
Configuration and Best Practices
1. Database Configuration
Ensure your PostgreSQL database is properly configured for advisory locks:
-- Check current lock status SELECT * FROM pg_locks WHERE locktype = 'advisory'; -- Set appropriate connection and statement timeouts SET statement_timeout = '30s'; SET lock_timeout = '10s';
2. Spring Configuration
Configure your PostgresLockService
bean:
@Configuration public class LockConfiguration { @Bean public LockService lockService(JdbcTemplate jdbcTemplate) { return new PostgresLockService(jdbcTemplate); } }
Key Benefits and Considerations
Benefits:
- Cluster-safe: Works across multiple JVM instances
- Transactional: Automatically releases locks on transaction completion
- Lightweight: No additional infrastructure required
- Reliable: Leverages PostgreSQL’s proven lock mechanisms
- Flexible: Supports both blocking and non-blocking approaches
Considerations:
- Database dependency: Requires PostgreSQL database connection
- Transaction requirement: Locks must be used within database transactions
- Lock key collision: Different lock names with same hash could collide
- Connection pooling: Consider impact on database connection pools
Conclusion
PostgreSQL advisory locks provide a robust foundation for implementing cluster-safe once-only methods in Java applications. The lock-postgres utility simplifies this implementation by providing a clean API that integrates seamlessly with Spring’s transaction management.
By using these distributed locks, you can ensure that critical operations execute exactly once across your entire cluster, preventing data inconsistencies and duplicate processing. The transaction-based approach ensures that locks are automatically cleaned up, even in failure scenarios, making your distributed system more reliable and maintainable.
Whether you’re processing financial transactions, generating reports, or coordinating data migrations, PostgreSQL advisory locks offer a battle-tested solution for distributed coordination without the complexity of additional infrastructure components.