Skip to content

Allow packed data to be captured while executing skip(), etc. - #23

Merged
methane merged 1 commit into
msgpack:masterfrom
jnothman:write_bytes
Dec 6, 2012
Merged

methane merged 1 commit into
msgpack:masterfrom
jnothman:write_bytes

Conversation

@jnothman

Copy link
Copy Markdown
Contributor

One may want to acquire the bytes corresponding to a single msgpack message. For example, consider a server whose job is to forward messages to a set of workers.

In the first instance, it may not care what the messages contain, it just reads messages off a stream and passes them on to workers. With the current msgpack-python, if the message length is not known, this requires unpacking and re-packing. Ideally one might want Unpacker.read_packed(). This patch instead offers the idiom:

bio = BytesIO()
unpacker.skip(bio.write)
msg_bytes = bio.getvalue()

Now consider the case that our server still passes on raw messages, but only those who have a particular value, say route_to == 'me':

bio = BytesIO()
obj = unpacker.unpack(bio.write)
if obj['route_to'] is 'me':
  route_to_me(bio.getvalue())

This avoids packing. With read_map_header supporting a similar interface, the following does the same, but avoids full unpacking:

bio = BytesIO()
route_to = None
map_size = unpacker.read_map_header(bio.write)
for i in range(map_size):
  key = unpacker.unpack(bio.write)
  if key is 'route_to':
    route_to = unpacker.unpack(bio.write)
  else:
    unpacker.skip(bio.write)
if route_to is 'me':
  route_to_me(bio.getvalue())

@jnothman

jnothman commented Oct 4, 2012

Copy link
Copy Markdown
Contributor Author

Updated to current HEAD.

methane added a commit that referenced this pull request Dec 6, 2012
Allow packed data to be captured while executing skip(), etc.
@methane
methane merged commit 54916f7 into msgpack:master Dec 6, 2012
@jnothman

jnothman commented Dec 6, 2012

Copy link
Copy Markdown
Contributor Author

Great! I look forward to using this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants