Post a non JSON event using HTTP API

EventStoreDB
#1

Hi,

I’m using the HTTP API to post events.

If I want to post a JSON event that’s documented:

curl -i -d@./temp/event.json “http://127.0.0.1:2113/streams/newstream” -H “Content-Type:application/json” -H “ES-EventType: SomeEvent” -H “ES-EventId: C322E299-CB73-4B47-97C5-5054F920746E”

But I can’t work out how to post a binary event:

curl -i -d@./temp/event.bin “http://127.0.0.1:2113/streams/newstream” -H “Content-Type:application/octet-stream” -H “ES-EventType: SomeEvent” -H “ES-EventId: C322E299-CB73-4B47-97C5-5054F920746E”

Says that the content type is not supported.

Posting a binary event as json seems to work but it sets the isJson flag to true so I’m worried it will break something.

Have I missed something?

cheers

Andrew

#2

So it will mark it as json (to be attempted to be deserialized in
projections). I have look look at the code but try text/xml (doesn't
set isjson)

#3

Hi Greg,

I tried xml three ways:

  1. using vnd.eventstore.events+xml with normal xml.

curl-i -d @event.xml “http://127.0.0.1:2113/streams/newstream” -H “Content-Type:application/vnd.eventstore.events+xml”

fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e4

event-type

1

This also came through as isJson=true

I also appears to convert to json when retrieving the event with that content type:

curl http://127.0.0.1:2113/streams/newstream/8 -H “Accept: application/vnd.eventstore.atom+json”

returned:

“content”: {

“eventStreamId”: “newstream”,

“eventNumber”: 8,

“eventType”: “event-type”,

“eventId”: “fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e4”,

“data”: {

“MyEvent”: {

“Something”: “1”

}

},

“metadata”: “”

},

  1. I had similar results when wrapping my event in a CDATA section.

  2. Using application/xml

curl -i -d @event.xml “http://127.0.0.1:2113/streams/newstream” -H “Content-Type:application/xml” -H “ES-EventType: blah” -H “ES-EventId: fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e1”

ES-EventType header was required but it used the EventType in my file. ES-EventId was also taken from the file.

This also came through with isJson = true.

The results for binary files seem to be base64 enocded and returned as a JSON string. Does that sound right?

cheers

Andrew

#4

Will have to take a look through it. Will do this evening.

#5

I am a bit confused looking through this. What version are you
running? From looking at the code
https://github.com/EventStore/EventStore/blob/release-v3.7.0/src/EventStore.Core/Services/Transport/Http/AutoEventConverter.cs#L89
it seems that XML is just being taken in as raw with isJson = false.

#6

There was a change to this recently BTW.

#7

Hi Greg,

Sorry was I was on 3.5. Now using 3.6.1.

If I post my event as application/xml:

curl -i -d @event.xml “http://127.0.0.1:2113/streams/newstream” -H “Content-Type:application/xml” -H “ES-EventType: blah” -H “ES-EventId: fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e1”

Then retrieve it has application/json:

curl -i http://127.0.0.1:2113/streams/newstream/0 -H “Accept: application/json”

In some cases though the file seems to have been munged by this process. I couldn’t round trip a binary file this way. A text file came back with newlines stripped.

Also using the HTTP interface this way there doesn’t seem to be a way to perform batch writes as you can with json (and binary using the C# client)?

cheers

Andrew

#8

When you post using application/xml what is in the stream after (eg if
viewing stream). I would guess the binary data with isjson=false?

#9

curl -i http://127.0.0.1:2113/streams/newstream?embed=body -H “Accept: application/json”

Returns events that look like:

{

“eventId”: “fbf4a1a1-b4a3-4dfe-a01f-ec52c34e16e1”,

“eventType”: “blah”,

“eventNumber”: 0,

“streamId”: “newstream”,

“isJson”: false,

“isMetaData”: false,

“isLinkMetaData”: false,

“positionEventNumber”: 0,

“positionStreamId”: “newstream”,

“title”: “0@newstream”,

“id”: “http://127.0.0.1:2113/streams/newstream/0”,

“updated”: “2016-05-09T23:52:52.589151Z”,

“author”: {

“name”: “EventStore”

},

“summary”: “blah”,

“links”: [

{

“uri”: “http://127.0.0.1:2113/streams/newstream/0”,

“relation”: “edit”

},

{

“uri”: “http://127.0.0.1:2113/streams/newstream/0”,

“relation”: “alternate”

}

]

}

#10

And a get to /newstream/0 with accept application/xml?

If you do it as json it will try to convert the data to json.

#11

curl -i http://127.0.0.1:2113/streams/newstream/0 -H “Accept: application/xml”

I see this message on the console:
[51143,16,10:59:57.593] Error while processing message EventStore.Core.Messages.ClientMessage+ReadEvent in queued handler ‘StorageReaderQueue #2’.
XmlNodeConverter can only convert JSON that begins with an object.

The request eventually returns: HTTP/1.1 408 Server was unable to handle request in time

#12

ok will take a look at it, looks like its trying to convert the data to xml.

I think support an octet media type for raw data would be useful anyways.

#13

Yep sure enough
https://github.com/EventStore/EventStore/blob/release-v3.7.0/src/EventStore.Core/Services/Transport/Http/AutoEventConverter.cs#L29
we will add another media type for octet data which gets no
conversions/parsing etc associated with it.

#14

For the batch write question. I’m wondering if we could have support for providing a base64 encoded string instead of a data object.

Something like:

[
    {
      "eventId"    : "string",
      "eventType"  : "string",
      "data"       : "object" | "binarydata"       : "base64 encoded string"
      "metadata"   : "object"
    }
]

If there is a binarydata element instead of a data element it could be stored as isJson = false. The downside of base64 encoding would the size of the request would be larger than required but the use cases I can imagine don’t require large events just not json/xml.

I’m happy to offer a PR for this change.

cheers

Andrew