NewsForumsCodeLogsGallerySpecsSearch

libcurl PUT

BlitzMax Forums/Brucey's Modules/libcurl PUT

Htbaa(Posted 2009) [#1]
I'm having a bit of trouble with doing a PUT request. Currently working on an interface to the REST API of Mosso's CloudFiles service.

By setting the following option I setup cURL to perform a PUT request.
curl.setOptString(CURLOPT_CUSTOMREQUEST, "PUT")


Then I want to set the content (just a plain string), doing so:
Local content:String = "Some text"
curl.setOptString(CURLOPT_POSTFIELDS, content)
curl.setOptLong(CURLOPT_POSTFIELDSIZE, content.Length)


With the following I set the appropriate content-type:
curl.httpHeaders(["Content-Type: text/plain"])


After sending the request and checking the results from VERBOSE it sets the Content-Type to application/x-www-form-urlencoded. Is this because of CURLOPT_POSTFIELDS?

I've checked the libcurl documentation and found the CURLOPT_PUT, CURLOPT_READDATA and CURLOPT_INFILESIZE options. The first one is just a boolean. But I'm not sure how and if I need to use the other 2 options.

My PUT request is being executed without problems and the Mosso CloudFiles service accepts it, but the content-type is being set to application/x-www-form-urlencoded which I don't want to happen.

Any thoughts? Code can be found here


Brucey(Posted 2009) [#2]
There's a C example here that might help :

http://curl.haxx.se/lxr/source/docs/examples/httpput.c

Unless you really need to post as a form?


Htbaa(Posted 2009) [#3]
No I don't need to post as a form :-). I'll try out the example later. Thanks!


Htbaa(Posted 2009) [#4]
I figured out how to use the callback for PUT operations, but I'm having trouble writing my content to the buffer.

In the example, fread() is being used. I can't use that as I'm (currently) simply passing a (c)string to my callback function. Do you happen to know how to append another c-string to a char * buffer?

Edit: Or is it as simple as this???
Function ReadCallback:Int(buffer:Byte Ptr, size:Int, nmemb:Int, data:Byte Ptr)
	Local bufferContent:String = String.FromCString(buffer)
	Local content:String = String.FromCString(data)
	
	MemFree(buffer)
	buffer = content.ToCString()
	Return content.Length
End Function



Brucey(Posted 2009) [#5]
I might do it something like this :
Type TDataBuffer

	Field cstring:Byte Ptr
	Field length:Int
	Field offset:Int
	
	Function Create:TDataBuffer(text:String)
		Local this:TDataBuffer = New TDataBuffer
		this.cstring = text.ToCString()
		this.length = text.length() + 1  ' assumes ASCII only
		Return this
	End Function
	
	Method Read:Int(buffer:Byte Ptr, size:Int)
		If offset < length Then
			Local count:Int = length - offset
			If count > size Then
				count = size
			End If
			
			MemCopy(buffer, cstring + offset, count)
			
			offset:+ count
			
			Return count
		End If
		
		' end of data
		Return 0
	End Method

	Method Delete()
		If cstring Then
			MemFree(cstring)
			cstring = Null
		End If
	End Method
	
End Type

Function read_callback:Int(buffer:Byte Ptr, size:Int, data:Object)

	Return TDataBuffer(data).Read(buffer, size)

End Function

I just knocked this together, so it might not even compile - let alone be entirely correct - but you should get the idea.
The problem I noticed was that you need to keep the "cstring" around if the buffer is smaller than the data you have to pass back. This is one way to do it.

HTH


Brucey(Posted 2009) [#6]
Or you could use ToUTF8String(), and get the length of that. strlen()


Htbaa(Posted 2009) [#7]
I figured I could also use TCurlEasy.setReadStream(). I convert my string to a TStream like this (example taken from here):

Local str:String = "My own special content :-)"
Local stream:TRamStream = CreateRamStream (str, SizeOf(str), True, False)


Checking if it was actually copied:
Print stream.ReadString(str.Length)


Here's my full example.

SuperStrict
Import bah.libcurl

Local curl:TCurlEasy = TCurlEasy.Create()
curl.setOptInt(CURLOPT_VERBOSE, 1)
curl.setOptInt(CURLOPT_FOLLOWLOCATION, 1)

curl.setOptString(CURLOPT_CUSTOMREQUEST, "PUT")
curl.setOptString(CURLOPT_URL, "http://192.168.0.10/dav/sometest.txt")

curl.setOptInt(CURLOPT_UPLOAD, True)
curl.setOptInt(CURLOPT_PUT, True)

Local str:String = "My own special content :-)"
Local stream:TRamStream = CreateRamStream (str, SizeOf(str), True, False)
curl.setOptLong(CURLOPT_INFILESIZE_LARGE, SizeOf(str))
Notify stream.ReadString(str.Length)
curl.setReadStream(stream)

Local res:Int = curl.perform()

Local errorMessage:String
If res Then
	errorMessage = CurlError(res)
	Print errorMessage
End If

curl.cleanup()
curl.freeLists()


With this example I'm getting a HTTP 417 error:
Expectation Failed
    The server cannot meet the requirements of the Expect request-header field.


Looking at the request it sends a Expect: 100-continue header. The next request that's being executed ends with a 417 error. So I started to dig into the source code of libcurl.mod and added some debugstops at TCurlEasy.readStreamCallback(). However, this method never gets called after setting it with TCurlEasy.setReadStream().

Also, looking at the output from cURL I don't see any content being placed into the request.

Am I forgetting something or is there some bug here?


Htbaa(Posted 2009) [#8]
I'm sorry to bother again but no matter what I do I can't get it to call TCurlEasy.readStreamCallback(). I've added some debuglog's in curlmain.bmx and it just isn't being called. I also took a look if the corresponding constants had the right values. All seemed right to me.

I'd really appreciate it if you could take a closer look.

Thanks.


Brucey(Posted 2009) [#9]
I hacked in direct Stream functionality to this example I posted a while ago.

Local stream:TStream = ReadStream("somefile....")

curl.setOptLong(CURLOPT_INFILESIZE_LARGE, stream.size())

curl.setReadStream(stream)

DebugLog stream.pos()

Local res:Int = curl.perform()

DebugLog stream.pos()


which gave me this output :
DebugLog:0
220 (vsFTPd 2.0.7)

331 Please specify the password.

230 Login successful.

257 "/home/brucey"

229 Entering Extended Passive Mode (|||35399|)

200 Switching to Binary mode.

150 Ok to send data.

DebugLog:28262


So... for FTP at least, we can see that it is working as expected - i.e. using the read stream callback stuff.

Of course, it doesn't help you with your current problem, but it does answer one of the questions.

I don't have anything set up locally to accept PUT via HTTP at the moment, so it may take a bit to put something together - my php is a bit rusty :-p
(but if you have a small snippet I can drop onto my web-server...)


Htbaa(Posted 2009) [#10]
I believe the HTTP server actually needs to be configured to accept PUT requests. Not sure though. I'm testing it with a VMWare session of Ubuntu 9.04-server using lighttpd with the webdav module. If you want I can post the configuration for it here.


Brucey(Posted 2009) [#11]
Googling around, it seems that it's a "bug" with lighttpd.

I set it up on my linux box, and it exhibits the same behaviour.

An old libcurl post here : http://curl.haxx.se/mail/archive-2005-11/0126.html

It appears that lighttpd returns a 417 regardless... so, the thing to do, seems to be to not "Expect:" at all.
You can do this by overriding the default header with this :
curl.httpHeader(["Expect:"])

which, if you have verbose debugging on, you'll see that the header disappears completely, and the PUT request performs as expected.
(well, it would if I had it configured properly, I suppose, but I'm getting a 404 after the data is sent to the server - which I guess is a lack of rights or something)

I also appear to have found a "bug", in that I'm using ReadBytes(), which throws an exception if you attempt to read past the end of the file - rather than returning 0. So, I think I'll change it to Read() instead of ReadBytes().

Sorry for the delay in getting to the bottom of this... but I hadn't really had time to set up all this until now.


Htbaa(Posted 2009) [#12]
Just had a quick look at it also with setting the header you suggested with lighttpd didn't make a difference for me. But then again, I also haven't really taken the time for it. I'll try to find some more time to look into it this week.

I need to get my Mosso CloudFiles module done :P


Brucey(Posted 2009) [#13]
Without
curl.httpHeader(["Expect:"])

libcurl will had something like :
Expect: 100-continue

Which lighttpd will never return, apparently.
Using the customised header will remove that Expect from the headers passed to the server, and the 417 error should go away.

Well, it did here, and that's what the libcurl emails suggested :-)


Htbaa(Posted 2009) [#14]
Finally had some time for it again :-). Decided to just use setReadStream(). After testing it actually reads the stream but I'm not seeing any data in my output window with CURLOPT_VERBOSE on.

So I checked the httpHeader() method, went to the processArray() function and it just iterates fine over my array with headers. However, my custom headers aren't showing up in the verbose output.

If httpHeader() is being called more then once it seems like any previous headers that were set are being reset. Is this supposed to happen?

Edit: Got it to work, finally :-)! But only after making a combined headers array and thus calling httpHeader() once.