Sorry, I should be clearer about the current state of the code, which for the Python extension is: currently completely broken. Since I was focusing on the core interfaces, the more peripheral pieces (like the language extensions) are totally broken at the moment.
> Certain abstractions really lost me, like pushing and pulling between sources and sinks.
Hopefully more documentation will make this clear. Making sources and sinks a general abstraction makes the event-based interface independent of any specific serialization (like protobuf binary format, protobuf text format, a JSON serialization of the same schema, etc). The key thing about protobufs is that a .proto file defines a typed tree structure, and the core interfaces of upb let you iterate over that tree structure, regardless of how exactly that tree structure was serialized.
> Why not just let a top-level event loop run the show in terms of buffered reads and size calculation for writes?
In my most recent interface, the upb_src does indeed run an event loop, and calls callbacks for every input value, or on a submessage start, or on a submessage end. For a while I wanted to make upb_src a pull-based interface instead, to give the application more control over the main loop, but this created more problems than it solved.
Sorry, I should be clearer about the current state of the code, which for the Python extension is: currently completely broken. Since I was focusing on the core interfaces, the more peripheral pieces (like the language extensions) are totally broken at the moment.
> Certain abstractions really lost me, like pushing and pulling between sources and sinks.
Hopefully more documentation will make this clear. Making sources and sinks a general abstraction makes the event-based interface independent of any specific serialization (like protobuf binary format, protobuf text format, a JSON serialization of the same schema, etc). The key thing about protobufs is that a .proto file defines a typed tree structure, and the core interfaces of upb let you iterate over that tree structure, regardless of how exactly that tree structure was serialized.
> Why not just let a top-level event loop run the show in terms of buffered reads and size calculation for writes?
In my most recent interface, the upb_src does indeed run an event loop, and calls callbacks for every input value, or on a submessage start, or on a submessage end. For a while I wanted to make upb_src a pull-based interface instead, to give the application more control over the main loop, but this created more problems than it solved.