Hi. I designed most of FlatBuffers, so let me see if I can clarify:
Clearly, FlatBuffers seeks a different design tradeoff than Cap'n Proto. We wanted to retain all the flexibility of Protobufs, while having the advantages of the "zero parsing / zero allocation" approach. That brought us to the vtable design.
In use cases where performance matters, i.e. you are churning through a large amount of data, you will be accessing the same vtable repeatedly, and the cost of the extra indirection (and the code associated with it) will be masked by the memory latency of the main data you're going through. It will be (close to) "for free". Even in cases where that isn't the case, whether or not it is able to match Cap'n Proto's accessor code is less interesting than it being endlessly faster than Protobuf and most other serializers out there, given that it is just as flexible.
vtables are de-duped on the fly during construction. It is typically not expensive.
Note this tiny overhead holds for "tables", we also have naked "structs", which do not have the indirection overhead, but have no forwards/backwards compatability. Useful for things like 2d/3d vector types, colors, and other small POD types that are used a lot.
FlatBuffers also has unions, which you should use instead of optionals when appropriate. We feel optionals have a lot of uses beyond just mere unions and forwards/backwards compatibility, however. Game objects can have a LOT of fields, many of which are often at their default value, and thus not stored on the wire. This gives significant compression. The zero-byte compression in Cap'n Proto is cool, but we prefer to not have to use additional buffers when reading. Optionals also give a lot of design freedom, i.e. you can add a field that you know is only needed for very few instances without fear of bloating your binaries, as an alternative to "subclassing", or indeed unions.
On field declaration order: to each his own, I guess. That said, the omission of explicit field id's is something that's handled at the level of the schema compiler, we could easily allow optional id declarations in the schema for people who want the flexibility of declaring things in an arbitrary order. Noted.
We should clean up the benchmark code, so it can be included, yes. It is not running a noop loop, we made sure of that. I haven't verified what makes Protobuf traversal that much slower, I am guessing it's causing a memory allocation somewhere.
Structs are a welcome improvement. Using Protobuf, I was forced to move from a 3d vector type to an array of doubles to get packed=true performance. Small loss of expressiveness, no big deal, but I can imagine it getting a lot worse for heterogeneous and/or more complex structs. Was a shock to see a dynamic allocation for every vector in my 1000-element array. (I understand why this is necessary given the spec of Protobuf though.)
Clearly, FlatBuffers seeks a different design tradeoff than Cap'n Proto. We wanted to retain all the flexibility of Protobufs, while having the advantages of the "zero parsing / zero allocation" approach. That brought us to the vtable design.
In use cases where performance matters, i.e. you are churning through a large amount of data, you will be accessing the same vtable repeatedly, and the cost of the extra indirection (and the code associated with it) will be masked by the memory latency of the main data you're going through. It will be (close to) "for free". Even in cases where that isn't the case, whether or not it is able to match Cap'n Proto's accessor code is less interesting than it being endlessly faster than Protobuf and most other serializers out there, given that it is just as flexible.
vtables are de-duped on the fly during construction. It is typically not expensive.
Note this tiny overhead holds for "tables", we also have naked "structs", which do not have the indirection overhead, but have no forwards/backwards compatability. Useful for things like 2d/3d vector types, colors, and other small POD types that are used a lot.
FlatBuffers also has unions, which you should use instead of optionals when appropriate. We feel optionals have a lot of uses beyond just mere unions and forwards/backwards compatibility, however. Game objects can have a LOT of fields, many of which are often at their default value, and thus not stored on the wire. This gives significant compression. The zero-byte compression in Cap'n Proto is cool, but we prefer to not have to use additional buffers when reading. Optionals also give a lot of design freedom, i.e. you can add a field that you know is only needed for very few instances without fear of bloating your binaries, as an alternative to "subclassing", or indeed unions.
On field declaration order: to each his own, I guess. That said, the omission of explicit field id's is something that's handled at the level of the schema compiler, we could easily allow optional id declarations in the schema for people who want the flexibility of declaring things in an arbitrary order. Noted.
We should clean up the benchmark code, so it can be included, yes. It is not running a noop loop, we made sure of that. I haven't verified what makes Protobuf traversal that much slower, I am guessing it's causing a memory allocation somewhere.