mirror of
https://github.com/Mahdi-zarei/nekoray.git
synced 2025-12-21 23:50:06 +08:00
daa6c93900
* refactor: migrate from protobuf to simple-protobuf * update go.mod * fix server * Update gRPC.cpp * Update gRPC.cpp
264 lines
10 KiB
Markdown
264 lines
10 KiB
Markdown
# simple-protobuf
|
|
|
|
[](https://github.com/tonda-kriz/simple-protobuf/actions/workflows/ci-linux-tests.yml)
|
|
[](https://github.com/tonda-kriz/simple-protobuf/actions/workflows/ci-windows-tests.yml)
|
|
[](https://github.com/tonda-kriz/simple-protobuf/actions/workflows/ci-macos-tests.yml)
|
|
[](https://tonda-kriz.github.io/simple-protobuf/library)
|
|
[](https://matrix.to/#/#simple-protobuf:gitter.im)
|
|
|
|
**simple data struct** serialization library for C++. With this library you can serialize and deserialize *POD* C++ structs directly to [JSON](https://json.org) or [protobuf](https://github.com/protocolbuffers/protobuf).
|
|
When used together with [etl library](https://github.com/ETLCPP/etl) it doesn't need to allocate any memory, so its suitable for embedded environments (see [extensions](doc/extensions.md)).
|
|
|
|
```CPP
|
|
namespace PhoneBook
|
|
{
|
|
struct Person {
|
|
enum class PhoneType : int32_t {
|
|
MOBILE = 0,
|
|
HOME = 1,
|
|
WORK = 2,
|
|
};
|
|
struct PhoneNumber {
|
|
// phone number is always required
|
|
etl::string<16> number;
|
|
std::optional< PhoneType > type;
|
|
};
|
|
std::optional< std::string > name;
|
|
// Unique ID number for this person.
|
|
std::optional< int32_t > id;
|
|
std::optional< std::string > email;
|
|
// all registered phones
|
|
std::vector< PhoneNumber > phones;
|
|
};
|
|
}// namespace PhoneBook
|
|
|
|
auto john = PhoneBook::Person{
|
|
.name = "John Doe",
|
|
.id = 1234,
|
|
.email = "jdoe@example.com",
|
|
};
|
|
//- serialize john to json-string
|
|
auto json = spb::json::serialize< std::string >( john );
|
|
//- deserialize john from json-string
|
|
auto person = spb::json::deserialize< PhoneBook::Person >( json );
|
|
//- serialize john to protobuf-vector
|
|
auto pb = spb::pb::serialize< std::vector< std::byte > >( john );
|
|
//- deserialize john from protobuf-vector
|
|
auto person2 = spb::pb::deserialize< PhoneBook::Person >( pb );
|
|
//- john == person == person2
|
|
```
|
|
|
|
## goal
|
|
|
|
goal of this library is to make [JSON](https://json.org) and [protobuf](https://github.com/protocolbuffers/protobuf) *part* of the C++ language itself.
|
|
|
|
## reason
|
|
|
|
There are literally a tons of [JSON](https://json.org) C++ libraries but most of them are designed in a way that the user needs to construct the json *Object* via some API and for serialization and deserialization there is a lot of boilerplate code like type/schema checking, `to_json`, `from_json`, macros... All this is needed to be done by the user, and it usually ends up with a conversion to some C++ struct.
|
|
|
|
spb works the other way around, from C++ struct to [JSON](https://json.org) or [protobuf](https://github.com/protocolbuffers/protobuf). With this approach user can focus only on the data, C++ struct, which is much more natural and spb will handle all the boring stuff like serialization/deserialization and type/schema checking.
|
|
|
|
## about
|
|
|
|
spb is an alternative implementation of [protobuf](https://github.com/protocolbuffers/protobuf) for C++. This is not an plugin for `protoc` but an **replacement** for `protoc`, so you don't need `protobuf` or `protoc` installed to use it. Serialization and deserialization to [JSON](https://json.org) or [protobuf](https://github.com/protocolbuffers/protobuf) is compatible with `protoc`, in other words, data serialized with code generated by `spb-protoc` can be deserialized by code generated by `protoc` and vice versa.
|
|
|
|
## usage
|
|
|
|
### dependencies
|
|
|
|
* C++ compiler (at least C++20)
|
|
* cmake
|
|
* std library
|
|
* *(optional) clang-format for code formatting*
|
|
|
|
### cheat sheet
|
|
|
|
```cmake
|
|
# add this repo to your project
|
|
add_subdirectory(external/spb-proto)
|
|
# compile proto files to C++
|
|
spb_protobuf_generate(PROTO_SRCS PROTO_HDRS ${CMAKE_SOURCE_DIR}/proto/person.proto)
|
|
# add generated files to your project
|
|
add_executable(myapp ${PROTO_SRCS} ${PROTO_HDRS})
|
|
# `spb-proto` is an interface library
|
|
# the main purpose is to update include path of `myapp`
|
|
target_link_libraries(myapp PUBLIC spb-proto)
|
|
```
|
|
|
|
### how to use
|
|
|
|
1. define a schema for you data in a `person.proto` file
|
|
|
|
```proto
|
|
package PhoneBook;
|
|
|
|
message Person {
|
|
optional string name = 1;
|
|
optional int32 id = 2; // Unique ID number for this person.
|
|
optional string email = 3;
|
|
|
|
enum PhoneType {
|
|
MOBILE = 0;
|
|
HOME = 1;
|
|
WORK = 2;
|
|
}
|
|
|
|
message PhoneNumber {
|
|
required string number = 1; // phone number is always required
|
|
optional PhoneType type = 2;
|
|
}
|
|
// all registered phones
|
|
repeated PhoneNumber phones = 4;
|
|
}
|
|
```
|
|
|
|
2. compile `person.proto` with `spb-protoc` into `person.pb.h` and `person.pb.cc`
|
|
|
|
```cmake
|
|
spb_protobuf_generate(PROTO_SRCS PROTO_HDRS ${CMAKE_SOURCE_DIR}/proto/person.proto)
|
|
```
|
|
|
|
*observe the beautifully generated `person.pb.h` and `person.pb.cc`*
|
|
|
|
```C++
|
|
namespace PhoneBook
|
|
{
|
|
struct Person {
|
|
enum class PhoneType : int32_t {
|
|
MOBILE = 0,
|
|
HOME = 1,
|
|
WORK = 2,
|
|
};
|
|
struct PhoneNumber {
|
|
// phone number is always required
|
|
std::string number;
|
|
std::optional< PhoneType > type;
|
|
};
|
|
std::optional< std::string > name;
|
|
// Unique ID number for this person.
|
|
std::optional< int32_t > id;
|
|
std::optional< std::string > email;
|
|
// all registered phones
|
|
std::vector< PhoneNumber > phones;
|
|
};
|
|
}// namespace PhoneBook
|
|
```
|
|
|
|
3. use `Person` struct natively and de/serialize to/from json/pb
|
|
|
|
```CPP
|
|
#include <person.pb.h>
|
|
|
|
auto john = PhoneBook::Person{
|
|
.name = "John Doe",
|
|
.id = 1234,
|
|
.email = "jdoe@example.com",
|
|
};
|
|
|
|
auto json = spb::json::serialize( john );
|
|
auto person = spb::json::deserialize< PhoneBook::Person >( json );
|
|
auto pb = spb::pb::serialize( john );
|
|
auto person2 = spb::pb::deserialize< PhoneBook::Person >( pb );
|
|
//- john == person == person2
|
|
```
|
|
|
|
## API
|
|
|
|
All generated messages (and enums) are using the following API [`include/spb/json.hpp`](include/spb/json.hpp) and [`include/spb/pb.hpp`](include/spb/pb.hpp)
|
|
|
|
```CPP
|
|
//- serialize message via writer (all other `serialize` are just wrappers around this one)
|
|
//- example: `auto serialized_size = spb::pb::serialize( message, my_writer );`
|
|
auto serialize( const auto & message, spb::io::writer on_write ) -> size_t;
|
|
|
|
//- return size in bytes of serialized message
|
|
//- example: `auto serialized_size = spb::pb::serialize_size( message );`
|
|
auto serialize_size( const auto & message ) -> size_t;
|
|
|
|
//- serialize message into container like std::string, std::vector, ...
|
|
//- example: `auto serialized_size = spb::pb::serialize( message, my_string );`
|
|
template < typename Message, spb::resizable_container Container >
|
|
auto serialize( const Message & message, Container & result ) -> size_t;
|
|
|
|
//- serialize message and return container like std::string, std::vector, ...
|
|
//- example: `auto my_string = spb::pb::serialize< std::string >( message );`
|
|
template < spb::resizable_container Container = std::string, typename Message >
|
|
auto serialize( const Message & message ) -> Container;
|
|
|
|
```
|
|
|
|
```CPP
|
|
//- deserialize message from reader (all other `deserialize` are just wrappers around this one)
|
|
//- example: `spb::pb::deserialize( message, my_reader );`
|
|
void deserialize( auto & message, spb::io::reader on_read );
|
|
|
|
//- deserialize message from container like std::string, std::vector, ...
|
|
//- example: `spb::pb::deserialize( message, my_string );`
|
|
template < typename Message, spb::size_container Container >
|
|
void deserialize( Message & message, const Container & protobuf );
|
|
|
|
|
|
//- return deserialized message from container like std::string, std::vector, ...
|
|
//- example: `auto message = spb::pb::deserialize< Message >( my_string );`
|
|
template < typename Message, spb::size_container Container >
|
|
auto deserialize( const Container & protobuf ) -> Message;
|
|
|
|
//- return deserialized message from reader
|
|
//- example: `auto message = spb::pb::deserialize< Message >( my_reader );`
|
|
template < typename Message >
|
|
auto deserialize( spb::io::reader reader ) -> Message;
|
|
```
|
|
|
|
API is prefixed with `spb::json::` for **json** and `spb::pb::` for **protobuf**,
|
|
template concepts [`spb::size_container`](include/spb/concepts.h) and [`spb::resizable_container`](include/spb/concepts.h) are defined in [`include/spb/concepts.h`](include/spb/concepts.h), [`spb::io::reader`](include/spb/io/io.hpp) and [`spb::io::writer`](include/spb/io/io.hpp) are user specified *functions* for IO, more info at [`include/io/io.hpp`](include/spb/io/io.hpp)
|
|
|
|
## type mapping
|
|
|
|
| proto type | CPP type | GPB encoding |
|
|
|------------|-------------|-------------|
|
|
| `bool` | `bool` | varint |
|
|
| `float` | `float` | 4 bytes |
|
|
| `double` | `double` | 8 bytes |
|
|
| `int32` | `int32_t` | varint |
|
|
| `sint32` | `int32_t` | zig-zag varint |
|
|
| `uint32` | `uint32_t` | varint |
|
|
| `int64` | `int64_t` | varint |
|
|
| `sint64` | `int64_t` | zig-zag varint |
|
|
| `uint64` | `uint64_t` | varint |
|
|
| `fixed32` | `uint32_t` | 4 bytes |
|
|
| `sfixed32` | `int32_t` | 4 bytes |
|
|
| `fixed64` | `uint64_t` | 8 bytes |
|
|
| `sfixed64` | `int64_t` | 8 bytes |
|
|
| `string` | `std::string` | utf8 string |
|
|
| `bytes` | `std::vector< std::byte >` | base64 encoded in json |
|
|
|
|
| proto type modifier | CPP type modifier | Notes |
|
|
|---------------------|-------------|-------------|
|
|
| `optional` | `std::optional<Message>` | |
|
|
| `optional` | `std::unique_ptr<Message>` | if there is cyclic dependency between messages ( A -> B, B -> A )|
|
|
| `repeated` | `std::vector<Message>` | |
|
|
|
|
See also [extensions](doc/extensions.md) for user specific types and advanced usage.
|
|
|
|
## example
|
|
|
|
navigate to the [example](example/) directory.
|
|
|
|
## status
|
|
|
|
* [x] Make it work
|
|
* [x] Make it right
|
|
* [ ] Make it fast
|
|
|
|
### roadmap
|
|
|
|
* [x] parser for proto files (supported syntax: `proto2` and `proto3`)
|
|
* [x] compile proto message to C++ data struct
|
|
* [x] generate json de/serializer for generated C++ data struct (serialized json has to be compatible with GPB)
|
|
* [x] generate protobuf de/serializer for generated C++ data struct (serialized pb has to be compatible with GPB)
|
|
|
|
### missing features
|
|
|
|
* RPC is not implemented
|
|
* extend is not implemented
|