diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..58324da --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,144 @@ +# Contribution Guide + +## First steps + +The project requires Go 1.24 or later. Clone the repository and install +dependencies. + +```sh +$ git clone https://github.com/tarantool/go-storage +$ cd go-storage +$ go mod tidy +``` + +## Running tests + +```bash +# To run unit and integration tests for the main package and each subpackage: +make test + +# To run tests with race detector: +make testrace + +# To run tests for a specific package (e.g., `driver/etcd`): +go test ./driver/etcd +``` + + +### Integration tests + +Integration tests require external dependencies: + +- **etcd**: Tests use etcd's embedded integration framework; no external setup + needed. +- **Tarantool Config Storage (TCS)**: Tests require Tarantool EE SDK. If the + SDK is not available, integration tests are skipped automatically. + +If you have Tarantool EE SDK installed, set `TARANTOOL_SDK_PATH` environment +variable to the SDK directory. The integration tests will automatically detect +it. + +### Code coverage + +To run tests with coverage collection: + +```bash +make coverage +``` + +This generates a coverage profile and prints coverage statistics. + +## Examples + +The project includes executable examples in the driver packages. To run the etcd +examples: + +```bash +go test -v -run Example ./driver/etcd +``` + +For TCS examples: + +```bash +go test -v -run Example ./driver/tcs +``` + +## Linting + +To run the linter (golangci-lint) on the codebase: + +```bash +make lint +``` + +This will automatically install golangci-lint if missing. The lint target also +runs `go mod tidy` and `go mod vendor` to ensure dependency consistency. + +## Formatting + +The project uses `goimports` for code formatting. The linter ensures formatting +consistency. You can format your code with: + +```bash +goimports -w . +``` + +## Spell checking + +To check for common misspellings in code and documentation: + +```bash +make codespell +``` + +## Code review checklist + +- Public API contains functions, variables, constants that are needed from + outside by users. All the rest should be left closed. +- Public functions, variables and constants contain at least a single-line + comment. +- Code is DRY (see "Do not Repeat Yourself" principle). +- New features have functional and probably performance tests. +- There are no changes in files not related to the issue. +- There are no obvious flaky tests. +- Commits with bugfixes have tests based on reproducers. +- Changelog entry is present in `CHANGELOG.md`. +- Public methods contain executable examples (contains a comment with reference + output). +- Autogenerated documentation looks good. Run `godoc -http=:6060` and point + your web browser to address "http://127.0.0.1:6060" for evaluating. +- Commit message header may start with a prefix with a short description + follows after colon. It is applicable to changes in a README, examples, + tests and CI configuration files. + Examples: `github-ci: add Tarantool 2.x-latest` and + `readme: describe how to run tests`. +- Check your comments, commit title, and even variable names to be + grammatically correct. Start sentences from a capital letter, end + with a dot. Everywhere - in the code, in the tests, in the commit + message. + +## Commit message guidelines + +Follow the [Conventional Commits](https://www.conventionalcommits.org/) style: + +- `feat:` for new features +- `fix:` for bug fixes +- `docs:` for documentation changes +- `test:` for test additions/modifications +- `refactor:` for code refactoring +- `chore:` for maintenance tasks + +Example: `feat: add support for etcd lease operations` + +## Pull request process + +1. Fork the repository and create a feature branch. +2. Ensure your code passes linting and tests. +3. Update the CHANGELOG.md with a brief description of your changes. +4. Submit a pull request with a clear description of the changes and their + purpose. + +## Questions? + +If you have questions, feel free to open an issue or reach out via Telegram +([English](http://telegram.me/tarantool) / [Russian](http://telegram.me/tarantoolru)). diff --git a/LICENCE b/LICENSE similarity index 100% rename from LICENCE rename to LICENSE diff --git a/README.md b/README.md index 7658ddc..16c1c49 100644 --- a/README.md +++ b/README.md @@ -3,12 +3,287 @@ [![Telegram EN][telegram-badge]][telegram-en-url] [![Telegram RU][telegram-badge]][telegram-ru-url] -

- -

- # go-storage: library to manage centralized configuration storages +### About + + + Tarantool Logo + + +**go-storage** is a Go library that provides a uniform interface for managing +centralized configuration storages, supporting multiple backends like etcd and +Tarantool Config Storage (TCS). It offers transactional operations, conditional +predicates, real-time watch, and data integrity features. + + +### Overview + +The library abstracts the complexities of different storage backends, providing +a consistent API for configuration management. It is designed for distributed +systems where configuration consistency, real-time updates, and transactional +safety are critical. + +### Features + +- Unified Storage Interface: Single API for multiple backend drivers (etcd, + TCS) +- Transactional Operations: Atomic transactions with conditional predicates +- Real-time Watch: Monitor changes to keys and prefixes +- Conditional Execution: Value and version-based predicates for safe updates +- Data Integrity: Built-in signing and verification of stored data +- Key‑Value Operations: Get, Put, Delete with prefix support +- Range Queries: Efficient scanning of keys with filters +- Extensible Drivers: Easy to add new storage backends + +### Installation + +```bash +go get github.com/tarantool/go-storage +``` + +### Quick Start + +#### Using etcd Driver + +```go +package main + +import ( + "context" + "log" + + "go.etcd.io/etcd/client/v3" + "github.com/tarantool/go-storage/driver/etcd" + "github.com/tarantool/go-storage/operation" +) + +func main() { + // Connect to etcd. + cli, err := clientv3.New(clientv3.Config{ + Endpoints: []string{"localhost:2379"}, + }) + if err != nil { + log.Fatal(err) + } + defer cli.Close() + + // Create etcd driver. + driver := etcd.New(cli) + + // Execute a simple Put operation. + ctx := context.Background() + _, err = driver.Execute(ctx, nil, []operation.Operation{ + operation.Put([]byte("/config/app/version"), []byte("1.0.0")), + }, nil) + if err != nil { + log.Fatal(err) + } +} +``` + +#### Using TCS Driver + +```go +package main + +import ( + "context" + "log" + + "github.com/tarantool/go-tarantool/v2" + "github.com/tarantool/go-storage/driver/tcs" + "github.com/tarantool/go-storage/operation" +) + +func main() { + // Connect to Tarantool. + conn, err := tarantool.Connect("localhost:3301", tarantool.Opts{}) + if err != nil { + log.Fatal(err) + } + defer conn.Close() + + // Create TCS driver. + driver := tcs.New(conn) + + // Execute a transaction. + ctx := context.Background() + resp, err := driver.Execute(ctx, nil, []operation.Operation{ + operation.Put([]byte("/config/app/name"), []byte("MyApp")), + }, nil) + if err != nil { + log.Fatal(err) + } + log.Printf("Transaction succeeded: %v", resp.Succeeded) +} +``` + +### Drivers + +#### etcd Driver +The `driver/etcd` package implements the storage driver interface for etcd. It +supports all etcd features including conditional transactions, leases, and +watch. + +#### TCS Driver +The `driver/tcs` package provides a driver for Tarantool Config Storage (TCS), +a distributed key‑value storage built on Tarantool. It offers high performance +and strong consistency. + +### API Overview + +#### Storage Interface +The core `Storage` interface (`storage.Storage`) provides high‑level methods: + +- `Watch(ctx, key, opts) <-chan watch.Event` – watch for changes +- `Tx(ctx) tx.Tx` – create a transaction builder +- `Range(ctx, opts) ([]kv.KeyValue, error)` – range query with prefix/limit + +#### Transaction Builder +The `tx.Tx` interface enables conditional transactions: + +```go +resp, err := storage.Tx(ctx). + If(predicate.ValueEqual(key, "old")). + Then(operation.Put(key, "new")). + Else(operation.Delete(key)). + Commit() +``` + +#### Operations +The `operation` package defines `Get`, `Put`, `Delete` operations. Each +operation can be configured with options. + +#### Predicates +The `predicate` package provides value and version comparisons: + +- `ValueEqual`, `ValueNotEqual` +- `VersionEqual`, `VersionNotEqual`, `VersionGreater`, `VersionLess` + +#### Watch +The `watch` package delivers real‑time change events. Watch can be set on a +single key or a prefix. + +#### Data Integrity with Typed Storage +The [`integrity`](https://pkg.go.dev/github.com/tarantool/go-storage/integrity) + package provides a high‑level `Typed` interface for storing and retrieving + values with built‑in integrity protection. It automatically computes hashes + and signatures (using configurable algorithms) and verifies them on + retrieval. + +##### Creating a Typed Storage Instance + +```go +package main + +import ( + "context" + "crypto/rand" + "crypto/rsa" + "log" + + clientv3 "go.etcd.io/etcd/client/v3" + "github.com/tarantool/go-storage" + "github.com/tarantool/go-storage/driver/etcd" + "github.com/tarantool/go-storage/hasher" + "github.com/tarantool/go-storage/crypto" + "github.com/tarantool/go-storage/integrity" +) + +func main() { + // 1. Create a base storage (e.g., etcd driver). + cli, err := clientv3.New(clientv3.Config{Endpoints: []string{"localhost:2379"}}) + if err != nil { + log.Fatal(err) + } + defer cli.Close() + + driver := etcd.New(cli) + baseStorage := storage.NewStorage(driver) + + // 2. Generate RSA keys (in production, load from secure storage). + privKey, err := rsa.GenerateKey(rand.Reader, 2048) + if err != nil { + log.Fatal(err) + } + + // 3. Build typed storage with integrity protection. + typed := integrity.NewTypedBuilder[MyConfig](baseStorage). + WithPrefix("/config"). + WithHasher(hasher.NewSHA256Hasher()). // adds SHA‑256 hash verification. + WithSignerVerifier(crypto.NewRSAPSSSignerVerifier(*privKey)). // adds RSA‑PSS signatures. + Build() + + // 4. Store a configuration object with automatic integrity data. + ctx := context.Background() + config := MyConfig{Environment: "production", Timeout: 30} + if err := typed.Put(ctx, "app/settings", config); err != nil { + log.Fatal(err) + } + + // 5. Retrieve and verify integrity. + result, err := typed.Get(ctx, "app/settings") + if err != nil { + log.Fatal(err) + } + + if result.Error != nil { + log.Printf("Integrity check failed: %v", result.Error) + } else { + cfg, _ := result.Value.Get() + log.Printf("Retrieved valid config: %+v", cfg) + } + + // 6. Range over all configurations under a prefix. + results, err := typed.Range(ctx, "app/") + if err != nil { + log.Fatal(err) + } + for _, res := range results { + log.Printf("Found config %s (valid: %v)", res.Name, res.Error == nil) + } +} + +type MyConfig struct { + Environment string `yaml:"environment"` + Timeout int `yaml:"timeout"` +} +``` + +##### Key Features +- Automatic Hash & Signature Generation: Values are stored together with + their hashes and/or signatures. +- Validation on read: `Get` and `Range` operations verify hashes and + signatures; invalid data is reported. +- Configurable Algorithms: Plug in any hasher (`hasher.Hasher`) and + signer/verifier (`crypto.SignerVerifier`). +- Prefix Isolation**: Each typed storage uses a configurable key prefix, + avoiding collisions. +- **Watch Support**: `Watch` method filters events for the typed namespace. + +The `integrity.Typed` builder also accepts custom marshallers (default is +YAML), custom namers, and separate signer/verifier instances for asymmetric +setups. + +### Examples + +Comprehensive examples are available in the driver packages: + +- **etcd examples**: [`driver/etcd/examples_test.go`](driver/etcd/examples_test.go) +- **TCS examples**: [`driver/tcs/examples_test.go`](driver/tcs/examples_test.go) + +Run them with `go test -v -run Example ./driver/etcd` or `./driver/tcs`. + +### Contributing + +Contributions are welcome! Please see the [CONTRIBUTING.md](CONTRIBUTING.md) +file for guidelines (if present) or open an issue to discuss your ideas. + +### License + +This project is licensed under the BSD 2‑Clause License – see the [LICENSE](LICENSE) file for details. + [godoc-badge]: https://pkg.go.dev/badge/github.com/tarantool/go-storage.svg [godoc-url]: https://pkg.go.dev/github.com/tarantool/go-storage [coverage-badge]: https://coveralls.io/repos/github/tarantool/go-storage/badge.svg?branch=master diff --git a/assets/logo.png b/assets/logo.png new file mode 100644 index 0000000..697d0b2 Binary files /dev/null and b/assets/logo.png differ