SoftClient4ES is a powerful SQL gateway for Elasticsearch. Query, manipulate, and manage your Elasticsearch data using familiar SQL syntax — through an interactive REPL client or as a Scala library.
Get started in seconds with the interactive SQL client:
Linux / macOS:
curl -fsSL https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/scripts/install.sh | bashWindows (PowerShell):
irm https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/scripts/install.ps1 | iexsoftclient4es --host localhost --port 9200-- Create a table (index)
CREATE TABLE users (
id KEYWORD,
name TEXT FIELDS(
raw KEYWORD
) OPTIONS (fielddata = true),
email KEYWORD,
age INTEGER,
created_at DATE,
PRIMARY KEY (id)
);
-- Insert data
INSERT INTO users (id, name, email, age)
VALUES ('1', 'Alice', 'alice@example.com', 30);
-- Query with SQL
SELECT name, email, age FROM users WHERE age > 25 ORDER BY name;
-- Update records
UPDATE users SET age = 31 WHERE id = '1';
-- Show tables
SHOW TABLES LIKE 'user%';| Feature | Benefit |
|---|---|
| 🗣️ SQL Interface | Use familiar SQL syntax — no need to learn Elasticsearch DSL |
| 🔄 Version Agnostic | Single codebase for Elasticsearch 6, 7, 8, and 9 |
| ⚡ Interactive REPL | Auto-completion, syntax highlighting, persistent history |
| 🔌 JDBC Driver | Connect from DBeaver, Tableau, or any JDBC-compatible tool |
| 🏹 Arrow Flight SQL | Zero-copy columnar access for DuckDB, Python, Apache Superset |
| 🔒 Type Safe | Compile-time SQL validation for Scala applications |
| 🚀 Stream Powered | Akka Streams for high-performance bulk operations |
| 🛡️ Production Ready | Built-in error handling, validation, and rollback |
| 📊 Materialized Views | Precomputed, auto-refreshed JOINs and aggregations |
CREATE TABLE products (
id KEYWORD,
name TEXT FIELDS(
raw KEYWORD
) OPTIONS (fielddata = true),
email KEYWORD,
price DOUBLE,
tags KEYWORD,
PRIMARY KEY (id)
);
ALTER TABLE products ADD COLUMN stock INTEGER;
DESCRIBE TABLE products;
DROP TABLE old_products;
TRUNCATE TABLE logs;INSERT INTO products (id, name, price) VALUES ('p1', 'Laptop', 999.99);
UPDATE products SET price = 899.99 WHERE id = 'p1';
DELETE FROM products WHERE price < 10;
COPY INTO products FROM '/data/products.json';SELECT name, price, COUNT(*) as sales
FROM products
WHERE category = 'electronics'
GROUP BY name, price
HAVING COUNT(*) > 10
ORDER BY sales DESC
LIMIT 100;Supported features: JOIN UNNEST, window functions, aggregations, nested fields, geospatial queries, and more.
Precomputed, automatically refreshed query results stored as Elasticsearch indices — ideal for denormalizing JOINs, precomputing aggregations, and enriching data across indices.
CREATE OR REPLACE MATERIALIZED VIEW orders_with_customers_mv
REFRESH EVERY 10 SECONDS
WITH (delay = '2s', user_latency = '1s')
AS
SELECT
o.id,
o.amount,
c.name AS customer_name,
c.email,
UPPER(c.name) AS customer_name_upper
FROM orders AS o
JOIN customers AS c ON o.customer_id = c.id
WHERE o.status = 'completed';
-- Query like a regular table
SELECT * FROM orders_with_customers_mv WHERE customer_name = 'Alice';
-- Inspect
DESCRIBE MATERIALIZED VIEW orders_with_customers_mv;
SHOW CREATE MATERIALIZED VIEW orders_with_customers_mv;
SHOW MATERIALIZED VIEW STATUS orders_with_customers_mv;Under the hood, materialized views orchestrate Elasticsearch transforms, enrich policies, ingest pipelines, and watchers — all generated from a single SQL statement.
📖 Materialized Views Documentation
Connect to Elasticsearch from any JDBC-compatible tool — DBeaver, Tableau, DataGrip, DbVisualizer, or any Java/Scala application.
Download the self-contained fat JAR for your Elasticsearch version:
| Elasticsearch Version | Artifact |
|---|---|
| ES 6.x | softclient4es6-jdbc-driver-0.1.3.jar |
| ES 7.x | softclient4es7-jdbc-driver-0.1.3.jar |
| ES 8.x | softclient4es8-jdbc-driver-0.1.3.jar |
| ES 9.x | softclient4es9-jdbc-driver-0.1.3.jar |
JDBC URL: jdbc:elastic://localhost:9200
Driver class: app.softnetwork.elastic.jdbc.ElasticDriver
Maven:
<dependency>
<groupId>app.softnetwork.elastic</groupId>
<artifactId>softclient4es8-jdbc-driver</artifactId>
<version>0.1.3</version>
</dependency>Gradle:
implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.1.3'sbt:
libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.1.3"The JDBC driver JARs are Scala-version-independent (no _2.12 or _2.13 suffix) and include all required dependencies.
Zero-copy columnar access to Elasticsearch over gRPC — for DuckDB, Python, Apache Superset, and any Arrow Flight SQL client.
docker run -p 32010:32010 \
-e ES_HOST=elasticsearch \
-e ES_PORT=9200 \
-e ES_USER=elastic \
-e ES_PASSWORD=changeme \
softnetwork/softclient4es8-arrow-flight-sql:latestimport adbc_driver_flightsql.dbapi as flight_sql
import duckdb
conn = flight_sql.connect("grpc://localhost:32010")
cursor = conn.cursor()
cursor.execute("SELECT * FROM ecommerce")
table = cursor.fetch_arrow_table() # zero-copy Arrow table
duckdb.sql("SELECT category, SUM(total_price) AS revenue FROM table GROUP BY category")docker compose --profile duckdb updocker compose --profile superset-flight updocker compose --profile grafana up| Apache Superset | Grafana |
|---|---|
 |
 |
📖 Arrow Flight SQL Documentation
📖 ADBC Driver Documentation
For programmatic access, add SoftClient4ES to your project.
| Elasticsearch | Artifact | Scala | JDK |
|---|---|---|---|
| 6.x | softclient4es6-jest-client |
2.12, 2.13 | 8+ |
| 6.x | softclient4es6-rest-client |
2.12, 2.13 | 8+ |
| 7.x | softclient4es7-rest-client |
2.12, 2.13 | 8+ |
| 8.x | softclient4es8-java-client |
2.12, 2.13 | 8+ |
| 9.x | softclient4es9-java-client |
2.13 only | 17+ |
// build.sbt
resolvers += "Softnetwork" at "https://softnetwork.jfrog.io/artifactory/releases/"
// Choose your Elasticsearch version
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es8-java-client" % "0.19.0"
// Add the community extensions for materialized views (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-community-extensions" % "0.1.3"
// Add the JDBC driver if you want to use it from Scala (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-jdbc-driver" % "0.1.3"import app.softnetwork.elastic.client._
val client = ElasticClientFactory.create()
// SQL queries
val results = client.search(SQLQuery("SELECT * FROM users WHERE age > 25"))
// Type-safe queries with compile-time validation
case class User(id: String, name: String, age: Int)
val users: Source[User, NotUsed] = client.scrollAs[User](
"SELECT id, name, age FROM users WHERE active = true"
)Automatically migrate index mappings with rollback support:
client.updateMapping("users", newMapping) // Handles backup, reindex, and rollbackStream millions of documents with backpressure handling:
client.bulkFromFile("/data/products.parquet", format = Parquet, idKey = Some("id"))Supported formats: JSON, NDJSON, Parquet, Delta Lake
Automatically selects the optimal strategy (PIT, search_after, or scroll):
client.scroll(SQLQuery("SELECT * FROM logs WHERE level = 'ERROR'"))
.runWith(Sink.foreach(processDocument))Seamlessly sync event-sourced systems with Elasticsearch.
| Topic | Link |
|---|---|
| REPL Client | 📖 Documentation |
| SQL Reference | 📖 Documentation |
| API Reference | 📖 Documentation |
| Materialized Views | 📖 Documentation |
| DDL Statements | 📖 Documentation |
| Arrow Flight SQL | 📖 Documentation |
| ADBC Driver | 📖 Documentation |
SoftClient4ES uses a dual-license model:
- Core (SQL engine, REPL client, Scala library) — Apache License 2.0 (open source)
- JDBC Driver, Arrow Flight SQL, ADBC Driver, and Materialized Views — Elastic License 2.0 (free to use, not open source)
| Feature | Community | Pro | Enterprise |
|---|---|---|---|
| Full SQL DDL (CREATE, ALTER, DROP TABLE) | Yes | Yes | Yes |
| Full SQL DML (INSERT, UPDATE, DELETE, COPY INTO) | Yes | Yes | Yes |
| Full SQL DQL (SELECT, JOIN UNNEST, aggregations, window functions) | Yes | Yes | Yes |
| Pipelines, Watchers, Enrich Policies | Yes | Yes | Yes |
| Interactive REPL client | Yes | Yes | Yes |
| Scala library (Akka Streams) | Yes | Yes | Yes |
| Elasticsearch 6, 7, 8, 9 support | Yes | Yes | Yes |
| JDBC driver (DBeaver, Tableau, etc.) | Yes | Yes | Yes |
| Arrow Flight SQL server + ADBC driver | Yes | Yes | Yes |
| Materialized Views (CREATE, REFRESH, DESCRIBE) | Max 3 | Limited | Unlimited |
| Priority support | - | - | Yes |
The JDBC driver and materialized views work on free/basic Elasticsearch clusters with the following exception:
| Elasticsearch Feature | Required ES License |
|---|---|
| Transforms (continuous data sync) | Free / Basic (ES 7.5+) |
| Enrich Policies (JOIN enrichment) | Free / Basic (ES 7.5+) |
| Watchers (auto-refresh enrich policies) | Platinum / Enterprise / Trial |
Materialized views with JOINs rely on Elasticsearch Watchers to automatically re-execute enrich policies when lookup table data changes. Without a Platinum ES license, this automation is unavailable — but an external scheduler (cron, Kubernetes CronJob, Airflow) can be used as a workaround. See the Materialized Views documentation for details.
- JDBC driver for Elasticsearch
- Materialized views with JOINs and aggregations
- Arrow Flight SQL server (gRPC, Docker)
- ADBC driver (in-process, columnar)
- Advanced monitoring dashboard
- Additional SQL functions
- ES|QL bridge
- Stop Rewriting Your Elasticsearch Code Every Version Upgrade
- Elasticsearch Queries That Never Break in Production
- It's 3 AM. Production Is Down. Your Only Tool Is curl.
- Elasticsearch Schema Management Was Hell. Then Someone Typed SQL.
- A 47-Line curl Script to Insert One Document. Seriously.
- Connect DBeaver to Elasticsearch. Yes, Really.
Contributions are welcome! See CONTRIBUTING.md for guidelines.
The core SQL engine and REPL client are licensed under the Apache License 2.0 — see LICENSE for details.
The JDBC driver, Arrow Flight SQL server, ADBC driver, and Materialized Views extension are licensed under the Elastic License 2.0 — free to use, not open source.
Built with ❤️ by the SoftNetwork team