GetInternetFile - downloads a file as a string

This code has been declared by its author to be Public Domain code.

GetInternetFile - downloads a file as a string by Zethrax

2013

Grabs a file from the internet and stores it in a global string variable.

Make sure you read the 'Requirements' and other sections of the comments.

Note that this function also requires the 'SplitString()' function found at: http://www.blitzbasic.com/codearcs/codearcs.php?code=2991

Can be used in combination with the 'SaveString' function linked below to download and save a file.

SaveString - saves a string value as a file
http://www.blitzbasic.com/codearcs/codearcs.php?code=3070

Function GetInternetFile( url$, dont_cache = True )
	; Downloads a file or webpage from the internet and places it into the string global 'G_internet_file$'.
	
	; = Parameters =
	; 'url$' - Should hold the full url of the file or webpage to be downloaded.
	; 'dont_cache' - A flag that controls whether browser caching is allowed.
	;	-	True = (default) The browser cache will be ignored and the file will always be sourced from the server.
	;	-	False = The file will be sourced from the browser cache if available, or from the server if not.

	; = Data Return Globals =
	; The HTTP file body (the actual file data) will be placed into 'G_internet_file$'.
	; The HTTP file header will be placed into 'G_internet_file_header$'.
	; The HTTP protocol version will be placed into 'G_internet_file_protocol_version$' (eg 'HTTP/1.0').
	; The HTTP status code will be placed into 'G_internet_file_status_code$' ('200' means no problems were encountered).
	; The HTTP status message will be placed into 'G_internet_file_status_message$' ('OK' means no problems were encountered).
	; The error codes returned by this function are also available in 'G_internet_file_error'.
	
	; = Function Return Values =
	; 0 = Success (no errors).
	; 1 = Unable to connect to server.
	; 2 = File request failed.
	; 3 = A stream error occurred.
	; 4 = The URL was badly formed and could not be parsed.
	; 5 = The file header was badly formed or corrupted.
	; 6 = Incorrectly formatted status line in header.
	; 7 = HTTP status code indicated an error.

	; = Requirements =
	; Requires that a string global named 'G_internet_file$' be declared.
	; Requires that a string global named 'G_internet_file_header$' be declared.	
	; Requires that a string global named 'G_internet_file_protocol_version$' be declared.
	; Requires that a string global named 'G_internet_file_status_code$' be declared.
	; Requires that a string global named 'G_internet_file_status_message$' be declared.
	; Requires that an integer global named 'G_internet_file_error' be declared.
	; Requires the 'SplitString()' function. (http://www.blitzbasic.com/codearcs/codearcs.php?code=2991)

	; = Reference Links =
	; <a href="http://en.wikipedia.org/wiki/List_of_HTTP_status_codes" target="_blank">http://en.wikipedia.org/wiki/List_of_HTTP_status_codes</a>
	; <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html" target="_blank">http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html</a>
	; <a href="http://www.jmarshall.com/easy/http/" target="_blank">http://www.jmarshall.com/easy/http/</a>

	Local tcp, startpos, endpos, host$, header_line$
	
	G_internet_file$ = ""
	G_internet_file_header$ = ""

	; Extract the hostname from the URL.
	startpos = Instr( url$, "://" )
	If startpos = 0 Then G_internet_file_error = 4 : Return 4
	startpos = startpos + 3
	endpos = Instr( url$, "/", startpos )
	If endpos = 0 Then endpos = Len( url$ ) + 1
	host$ = Mid( url$, startpos, endpos - startpos )
	If host$ = "" Then G_internet_file_error = 4 : Return 4
	
	tcp = OpenTCPStream( host$, 80 )
	
	If Not tcp Then G_internet_file_error = 1 : Return 1 ; Unable to connect to server.
	
	; Write the HTTP request header.
	; Note that 'WriteLine' automatically adds a linebreak (CRLF), so no need to add one.
	WriteLine tcp,"GET " + url$ + " HTTP/1.0"
	WriteLine tcp, "Host: " + host$
	If dont_cache
		WriteLine tcp, "Cache-Control: no-cache, must-revalidate"
		WriteLine tcp, "Expires: Sat, 26 Jul 1997 05:00:00 GMT" ; (Just an old date)
	EndIf
	WriteLine tcp, "User-Agent: downloader"
	WriteLine tcp, "Connection: close"
	WriteLine tcp, ""
	
	If Eof( tcp ) Then CloseTCPStream tcp : G_internet_file_error = 2 : Return 2 ; File request failed.

	header_line$ = ReadLine( tcp )
	If SplitString( header_line$, " " ) <> 2 Then CloseTCPStream tcp : G_internet_file_error = 6 : Return 6 ; Incorrectly formatted status line in header.
	G_internet_file_protocol_version$ = A_string$( 0 )
	G_internet_file_status_code$ = A_string$( 1 )
	G_internet_file_status_message$ = A_string$( 2 )
	If G_internet_file_status_code$ <> "200" Then CloseTCPStream tcp : G_internet_file_error = 7 : Return 7 ; HTTP status code indicated an error.
	G_internet_file_header$ = G_internet_file_header$ + header_line$ + G_eol$

	While Not Eof( tcp )
		header_line$ = ReadLine( tcp )
		If header_line$ = "" Then Exit ; The blank line terminating the header was found.
		G_internet_file_header$ = G_internet_file_header$ + header_line$ + G_eol$
	Wend 
	
	Select Eof( tcp )
		Case -1
			CloseTCPStream tcp
			G_internet_file_error = 3
			Return 3 ; A stream error occurred.
		Case 1
			CloseTCPStream tcp
			G_internet_file_error = 5
			Return 5 ; The file header was badly formed or corrupted.
	End Select		
	
	While Not Eof( tcp )
		G_internet_file$ = G_internet_file$ + Chr( ReadByte( tcp ) )
	Wend
	
	If Eof( tcp ) = -1 Then CloseTCPStream tcp : G_internet_file_error = 3 : Return 3 ; A stream error occurred.
	
	CloseTCPStream tcp
	
	G_internet_file_error = 0
	Return 0
End Function

The code can be found at: http://www.blitzbasic.com/codearcs/codearcs.php?code=3069

Comments

None.

Code Archives Forum