Scope
Elasticsearch is a distributed search and analytics engine, and Spring Data Elasticsearch lets a Spring Boot application read and write it through repositories, the same way it uses a database.
The interesting question is not how to call Elasticsearch, but what happens to your application when Elasticsearch is slow or down. Our approach keeps the two apart: the application writes each event to a buffer table in its own database, in the same transaction as the business change, and a scheduled job moves the buffered events to Elasticsearch every couple of minutes. An Elasticsearch outage then delays indexing; it never fails a user’s transaction, and nothing is lost.
If the pattern sounds familiar, it is a transactional outbox.
Tested with: Spring Boot 2.7.18, Spring Data Elasticsearch 4.4 and the Elasticsearch 7.17 Java client. See “Newer versions” at the end for Spring Boot 3.
Integration with Elasticsearch
Maven dependencies
Use the starter and let Spring Boot choose the version, so Spring Data Elasticsearch and the Elasticsearch client stay in the combination Spring Boot was tested with:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
Check the compatibility matrix against your Elasticsearch server version before you upgrade either side.
The scheduler runs on every instance of the application, so we use ShedLock to make sure only one instance runs it at a time. ShedLock is not managed by Spring Boot, so it needs a version:
<dependency>
<groupId>net.javacrumbs.shedlock</groupId>
<artifactId>shedlock-spring</artifactId>
<version>4.44.0</version>
</dependency>
<dependency>
<groupId>net.javacrumbs.shedlock</groupId>
<artifactId>shedlock-provider-jdbc-template</artifactId>
<version>4.44.0</version>
</dependency>
Elasticsearch client configuration
@Configuration
@EnableElasticsearchRepositories(basePackages = "com.example.activity.search")
public class ElasticsearchConfig {
private final Environment environment;
public ElasticsearchConfig(Environment environment) {
this.environment = environment;
}
@Bean
public RestHighLevelClient elasticsearchClient() {
try {
CredentialsProvider credentials = new BasicCredentialsProvider();
credentials.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(
environment.getRequiredProperty("elasticsearch.username"),
environment.getRequiredProperty("elasticsearch.password")));
// Trust the cluster's CA — and nothing else. Hostname verification stays on.
SSLContext sslContext = SSLContexts.custom()
.loadTrustMaterial(
new File(environment.getRequiredProperty("elasticsearch.truststore.path")),
environment.getRequiredProperty("elasticsearch.truststore.password").toCharArray())
.build();
RestClientBuilder builder = RestClient
.builder(new HttpHost(
environment.getRequiredProperty("elasticsearch.host"),
environment.getRequiredProperty("elasticsearch.port", Integer.class),
"https"))
.setHttpClientConfigCallback(http -> http
.setSSLContext(sslContext)
.setDefaultCredentialsProvider(credentials))
.setRequestConfigCallback(request -> request
.setConnectTimeout(5_000)
.setSocketTimeout(120_000));
return new RestHighLevelClientBuilder(builder.build())
.setApiCompatibilityMode(true) // lets the 7.17 client talk to an 8.x cluster
.build();
} catch (Exception e) {
throw new IllegalStateException("Could not create the Elasticsearch client", e);
}
}
@Bean
public ElasticsearchOperations elasticsearchTemplate(RestHighLevelClient elasticsearchClient) {
return new ElasticsearchRestTemplate(elasticsearchClient);
}
@Bean
public String activityIndexName() {
return environment.getRequiredProperty("elasticsearch.activity-index");
}
}
And in application.properties (secrets from the environment, never committed):
elasticsearch.host=search.example.com
elasticsearch.port=9200
elasticsearch.username=${ES_USERNAME}
elasticsearch.password=${ES_PASSWORD}
elasticsearch.truststore.path=/etc/app/elasticsearch-truststore.p12
elasticsearch.truststore.password=${ES_TRUSTSTORE_PASSWORD}
elasticsearch.activity-index=activity-log
The truststore holds the certificate authority that signed the cluster’s certificate (with a default Elasticsearch 8 installation, the http_ca.crt it generates). You can create one with keytool -importcert -alias es-ca -file http_ca.crt -keystore elasticsearch-truststore.p12 -storetype PKCS12.
Never ship a client that trusts every certificate. A trust-all
SSLContextplusNoopHostnameVerifierturns HTTPS into unauthenticated encryption: anyone who can sit between the application and the cluster can read the credentials and the data. If you need it for a throw-away local cluster, keep it in a local-only profile.
@EnableElasticsearchRepositories tells Spring Data where the Elasticsearch repositories live. Keep them in their own package, apart from the JPA repositories, so each Spring Data module only scans its own.
The buffer entity and the document
The buffer table only needs the event as a JSON string, plus the bookkeeping for retries:
@Entity
@Table(name = "activity_log_buffer")
@Getter
@Setter
@NoArgsConstructor
public class ActivityLogBuffer {
@Id
@SequenceGenerator(name = "activity_log_buffer_seq", sequenceName = "activity_log_buffer_seq", allocationSize = 1)
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "activity_log_buffer_seq")
private Long id;
@Column(name = "body")
private String body; // the event, as JSON
@Column(name = "last_error")
private String lastError;
@Column(name = "last_error_at")
private Instant lastErrorAt;
public ActivityLogBuffer(String body) {
this.body = body;
}
}
The document is the typed view of the same event, as it lives in Elasticsearch:
@Document(indexName = "#{@activityIndexName}", createIndex = false)
@Getter
@Setter
@NoArgsConstructor
public class ActivityLog {
@Id
private String id; // the buffer row's id: makes every write idempotent
private Long actorId;
private String action;
private String actionTimestamp; // ISO-8601; mapped as "date" in the index
private Long taskId;
private String taskType;
private String taskStatus;
}
About createIndex = false: with it, Spring Data never creates the index or sends its mapping. The index is created either by you (an index template or a script, with an explicit mapping — the recommended way for anything you search on) or, if the cluster allows it, by Elasticsearch itself when the first document arrives, with a dynamically guessed mapping. Either way, @Field annotations on the class do not reach the index; if you want Spring Data to own the mapping, use createIndex = true and let it create the index at startup.
Repositories
public interface ActivityLogBufferRepository extends JpaRepository<ActivityLogBuffer, Long> {
List<ActivityLogBuffer> findTop500ByOrderByIdAsc();
}
public interface ActivityLogRepository extends ElasticsearchRepository<ActivityLog, String> {
Page<ActivityLog> findByActorIdAndActionIn(Long actorId, List<String> actions, Pageable pageable);
}
ActivityLogRepository works like any Spring Data repository: derived queries, paging and sorting, translated into Elasticsearch queries through the client configured above.
Writing to the buffer
Wherever the application performs an action worth indexing, it saves a buffer row inside the same transaction as the change itself:
activityLogBufferRepository.save(new ActivityLogBuffer(objectMapper.writeValueAsString(event)));
If the business transaction rolls back, the event disappears with it; if it commits, the event is guaranteed to be indexed eventually.
Scheduling the transfer
Enable scheduling and ShedLock once:
@Configuration
@EnableScheduling
@EnableSchedulerLock(defaultLockAtMostFor = "PT10M")
public class SchedulerConfig {
@Bean
public LockProvider lockProvider(DataSource dataSource) {
return new JdbcTemplateLockProvider(JdbcTemplateLockProvider.Configuration.builder()
.withJdbcTemplate(new JdbcTemplate(dataSource))
.usingDbTime() // all instances use the database clock
.build());
}
}
ShedLock keeps its locks in a table that you create once (Oracle syntax shown):
CREATE TABLE shedlock (
name VARCHAR2(64 CHAR) NOT NULL,
lock_until TIMESTAMP(3) NOT NULL,
locked_at TIMESTAMP(3) NOT NULL,
locked_by VARCHAR2(255 CHAR) NOT NULL,
CONSTRAINT shedlock_pk PRIMARY KEY (name)
);
The scheduled job reads a bounded batch, outside any transaction, and hands each row to a second bean:
@Scheduled(cron = "${scheduler.activity-log.cron}")
@SchedulerLock(name = "activityLogToElasticsearch", lockAtLeastFor = "PT30S", lockAtMostFor = "PT2M")
public void sendActivityLogs() {
for (ActivityLogBuffer entry : activityLogBufferRepository.findTop500ByOrderByIdAsc()) {
activityLogSender.send(entry);
}
}
Each row is sent in its own transaction, so one failure does not stop the others:
@Service
public class ActivityLogSender {
// constructor injection of the two repositories and the ObjectMapper omitted
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void send(ActivityLogBuffer entry) {
try {
ActivityLog document = objectMapper.readValue(entry.getBody(), ActivityLog.class);
document.setId(String.valueOf(entry.getId()));
activityLogRepository.save(document); // same id on a retry: overwrites, never duplicates
activityLogBufferRepository.delete(entry);
} catch (Exception e) {
entry.setLastError(e.getClass().getSimpleName());
entry.setLastErrorAt(Instant.now());
activityLogBufferRepository.save(entry); // stays in the buffer; retried on the next run
}
}
}
Two details matter here. send lives in a separate bean: a @Transactional method called from the same class bypasses the Spring proxy, and REQUIRES_NEW would silently do nothing. And the document id is the buffer row’s id, so if the application crashes after Elasticsearch accepted the document but before the buffer row was deleted, the next run writes the same document again instead of a duplicate.
Reading
public Page<ActivityLog> findActivity(Long actorId, List<String> actions, Pageable pageable) {
return activityLogRepository.findByActorIdAndActionIn(actorId, actions, pageable);
}
Pitfalls
- Trust-all TLS. See the warning above. It is the most copied snippet in Elasticsearch tutorials, and the most dangerous.
- Unbounded batches. If Elasticsearch is down for a day, the buffer grows. Read a bounded batch per run (
findTop500…), neverfindAll(). - Self-invocation.
REQUIRES_NEWonly works through the proxy — keep the per-row method in another bean. - Mapping drift. With
createIndex = false, the index mapping lives outside the code. Keep the index template under version control next to the application, or let Spring Data create the index. - Version pinning. Pinning
spring-data-elasticsearchor the client by hand under a Spring Boot parent overrides the tested combination. Upgrade Spring Boot instead. - Personal data. An activity index usually records who did what. It is personal data in a second store: give the index a retention policy (index lifecycle management) and include it in your erasure process. Store an internal actor id rather than names or e-mail addresses.
Newer versions
The RestHighLevelClient used above is deprecated since Elasticsearch 7.15 and is not part of the 8.x clients. Spring Data Elasticsearch 5 (Spring Boot 3) is built on the new Elasticsearch Java API Client, so the client configuration changes — see the Spring Data Elasticsearch client documentation. The rest of the pattern — buffer table, ShedLock-guarded scheduler, one transaction per row, buffer id as document id — stays exactly the same.
Conclusion
Integrating Elasticsearch is the easy part. Keeping it from becoming a single point of failure is the part worth designing: write events to a buffer table in the business transaction, move them with a scheduled, cluster-safe job, and make every write idempotent. Elasticsearch can then go down for maintenance without anyone noticing anything but a short indexing delay.
Cheers!
Related sources
- Spring Data Elasticsearch: versions and compatibility
- Spring Data Elasticsearch: clients
- Spring Data Elasticsearch: object mapping
- ShedLock
- Elasticsearch Java API Client
- A Spring Boot resource server on Keycloak, done right: roles, ownership checks and identity - November 13, 2026
- GDPR-compliant logging in Spring Boot, part 3: the personal-data leaks outside your own code - October 19, 2026
- GDPR-compliant logging in Spring Boot, part 2: a build-time test that fails on personal data in log statements - October 19, 2026

