[Solved]How to get a redirected URL?

BlitzMax Forums/BlitzMax Programming/[Solved]How to get a redirected URL?

Grisu(Posted 2017) [#1]
Hi everyone!

Is there a way to get a redirected URL? My issue is that Fmod doesn't support this, so in some cases I can't stream an URL because the server redirects the connection to somewhere else and adds tokens to the end of the url.

If I add the main URL manually to my browser window it works fine. If I use the same new URL in my player via (CreateStreamURL(..)) it works too. But only for my session and not for other users.

Example URLs I got from ambushradio.gr:

MP3 (redirected)
http://stream.radiojar.com/k0kda1hmb
Compatible with: Windows Media Player, WinAmp, iTunes, VLC.

In addition it would nice, if I could automatically extract urls from the playlists. I don't, know if there are modules that do this?
M3U (playlist file)
http://stream.radiojar.com/k0kda1hmb.m3u

PLS (playlist file)
http://stream.radiojar.com/k0kda1hmb.pls
The most popular streaming format, compatible with most media players.


Example app (requires Brucey's Fmod module and MaxGUI):



Thanks in advance!
Grisu


xlsior(Posted 2017) [#2]
Not sure if this works in your context, but you can communicate directly to port 80 of the server and query it. I don't remember how to do that from within blitzmax, but know that you can do it. Here's a sample with telnet, to show you a proof of concept:

TELNET stream.radiojar.com 80

then send the following text:

GET /k0kda1hmb.pls HTTP/1.1
HOST: stream.radiojar.com
then send two enters
(hostname is necessary since most modern servers will allow countless websites to share the same IP. If you don't specify the host, then the default website listening on that port will answer, which may not be the one you're trying to reach)

in this case, the server responds with:


HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: audio/x-scpls
X-Cloud-Trace-Context: 8c37b5d7545f46238070b04a07281783
Date: Sun, 08 Jan 2017 10:11:09 GMT
Server: Google Frontend
Content-Length: 113

[Playlist]
NumberOfEntries=1
File1=http://162.13.25.174/k0kda1hmb.mp3
Title1=k0kda1hmb
Length=-1
Version=2



I'm assuming that this line contains the relevant info for you:
File1=http://162.13.25.174/k0kda1hmb.mp3

If I try to navigate directly to that URL in my browser, it starts playing the MP3 immediately.



---

NOTE: Trying the .m3u link you posted, the response showed the following location: http://162.13.25.174/k0kda1hmb

then telnetting to 162.13.25.174 port 80 and requesting k0kda1hmb , it returned the following:

HTTP/1.1 302 Found
location: http://ample-02.radiojar.com:80/k0kda1hmb?rj-ttl=5&rj-token=AAABWX2gmAWSiBc8WiNfR_g6-L2i5OW-QsbEblJ2MFUx9zs0J6ENIQ
Transfer-Encoding: chunked
Connection: Keep-Alive



the HTTP 302 means a temporary redirect (301 is permanent redirect), and the next line shows where the redirected location is. Pulling up that ample-02.radiojar.com URL plays the MP3 directly in my browser.


Grisu(Posted 2017) [#3]
Already had used this URL in my example code above. Sadly, it doesn't work.

You need some sort of "token sting" at the end, else it won't start streaming, i.e. http://ample-02.radiojar.com/k0kda1hmb.mp3?rj-ttl=5&rj-token=AAABWX2iWGq0oQU-CxNjopVaLLvpyP_u4M6wObJEZXfUR4tBoX8R0Q


xlsior(Posted 2017) [#4]
Updated post above.


Grisu(Posted 2017) [#5]
Same result with: http://162.13.25.174/k0kda1hmb. :(

If I copy and paste the full url from the browser window it works, but no other user can use it. Somehow VLC and other players get around this.

I would need to get the "final" url and use this for the OpenStreamURL() function instead.


Derron(Posted 2017) [#6]
@ grisu
Did you check that part of xlsior's bigger post? There the token was assigned - so you could build that url.

I think the other players support "sessions" (like "curl" would do) so the token is transported correctly and you do not need to extract it on your own.


bye
Ron


Grisu(Posted 2017) [#7]
I read the post, but I don't know how I could write a code that gives me that information. Adding a function to add the token to the streaming URL should be easy.

So I need to check, if I get a HTTP 302 (redirected) or 301 (static) response from an url. Then get the new URL +token for 302. And use the whole string in my player.

Just an idea: Can't I use libcurl() to do the job?


Henri(Posted 2017) [#8]
Hi Grisu,

you can use libCurl for the task. I'm just guessing that simply copying the redirect url with token isn't enough. The answer would be to process libCurls write-function directly to get the valid stream (just guessing :-)).
The url information is found in header.

-Henri


BlitzSupport(Posted 2017) [#9]
This Code Archives example checks for redirected URLs:

BlitzGet MaxDeluxe


Henri(Posted 2017) [#10]
For fun, here is a libCurl version of getting the header and possible processing the stream file (prototype). This does the same thing as James's example.

SuperStrict

Framework BaH.libcurl
Import brl.System
Import brl.StandardIO

Local curl:TCurlEasy = TCurlEasy.Create()

curl.setOptInt(CURLOPT_HEADER, 1)
curl.setOptString(CURLOPT_URL, "http://162.13.25.174/k0kda1hmb.mp3")
curl.setHeaderCallback( processHeader )

Local res:Int = curl.perform()

curl.cleanup()

Function processHeader:Int(buffer:Byte Ptr, size:Int, data:Object)
		
	Local s:String = String.FromBytes(buffer, size).Trim()
	
	'Check header start
	If s.StartsWith("HTTP/")
		Local pos:Int = s.Find(" ")
		Local code:Int = Int( s[pos..size].Trim() )
		
		DebugLog code
	EndIf
	
	Local pos:Int
	
	'Get location part of the header
	If s.StartsWith("location:")
		Local pos:Int = 10
		
		Local content:String = s[pos..size].Trim()
		
		DebugLog content
	EndIf
	
	Return size
EndFunction

Function processDownload:Int(buffer:Byte Ptr, size:Int, data:Object)
	
	Print "Start writing..." + size
	
	If Not size Then Return size
			
	If Not TStream(data) Then Notify("Error: WriteStream not found.", True); Return False
	'If Not fStream Then fStream = WriteStream(fName)
	TStream(data).WriteBytes(buffer, size)
	
	Return size
EndFunction


Playlist files are simple text files which you can easily extract the url / file info by parsing them line at a time seeking keywords.

-Henri


Grisu(Posted 2017) [#11]
Thanks all for the help. I went with Henri's code.

With the following functions I precheck an url to find out, if it's a redirected one. If it is, it should replace the "station_url" string with the new url. But only then.

I can't figure out how to make this work 100%. There are some urls that "break" the function. :(

Function Get_Redirected_Url(station_url:String)
Local curl:TCurlEasy = TCurlEasy.Create()

curl.setOptInt(CURLOPT_HEADER, 1)
curl.setOptString(CURLOPT_URL, station_url)
curl.setHeaderCallback( processHeader )
Local res:Int = curl.perform()
curl.cleanup()

End Function

Function ProcessHeader:Int(buffer:Byte Ptr, size:Int, data:Object)
' Used to find redirected station urls!		
		Local s:String = String.FromBytes(buffer, size).Trim()
		Local redir:Int=0
		
		'Check header start
		If s.StartsWith("HTTP/")
			Local pos:Int = s.Find(" ")
			Local code:Int = Int( s[pos..size].Trim() )
			'Print code 
			'DebugLog code
            If code <> 302 Then Return 0
        EndIf 
		
		Local pos:Int	
		'Get location part of the header
		If s.StartsWith("location:")
			Local pos:Int = 10
			Local content:String = s[pos..size].Trim()
             station_url:String=content
'            Print station_url 
			'DebugLog "> "+content
		EndIf
        Return size 
EndFunction



Grisu(Posted 2017) [#12]
I switched over to James code. Less fancy, but it seems to work better. ;)

Any way to "improve" this code further? Please let me know.

Function GetMovedUrL:Int(url:String)

	If Not url Then Return False
	
	Local success:Int = False	' File 302 with new url
	Local done:Int = False		' Exit loop (for retries, etc)
	
	Local host:String
	Local file:String

	Local bytestoread:Int
	Local date:String
	Local server:String
	Local contenttype:String
	Local location:String
	Local pos:Int

	Repeat
	
		If Left (url, 7) = "http://"
			url= Right (url, Len (url) - 7)
		EndIf
		
		Local slash:Int = Instr (url, "/")
	
		If slash
			host = Left (url, slash - 1)
			file = Right (url, Len (url) - slash + 1)
		Else
			host = url
			file = "/"
		EndIf


		Local http:TSocket = CreateTCPSocket ()
		
		If http
		
			If ConnectSocket (http, HostIp (host), 80)
				
				Local www:TStream = CreateSocketStream (http)
				
				WriteLine www, "GET " + file + " HTTP/1.1" ' "GET /" gets default page...
				WriteLine www, "Host: " + host
				WriteLine www, "User-Agent: BlitzGet Deluxe"
				WriteLine www, "Accept: */*"
				WriteLine www, ""
	
				Local response:String = ReadLine (www)
		
				'Print "Server response: " + response
				
				Local replycode:String
				
				If Left (response, 5) = "HTTP/"
					pos = Instr (response, " ")
					replycode = Mid (response, pos + 1, 3)
				EndIf
	
				Local header:String
	
				Repeat
			
					header = ReadLine (www)
			
					Local reply:String = ""
	
					pos = Instr (header, ": ")
	
					If pos
						reply = Left (header, pos + 1)
					EndIf
			
					Select Lower (reply)
						Case "content-length: "
							bytestoread = Int (Right (header, Len (header) - Len (reply)))
						Case "location: "
							location = Right (header, Len (header) - Len (reply))
							'Print location 
					End Select
	
					'If header Then 'Print header ' Skip blank line (if header = "" then nothing is printed)...
									
				Until header$ = "" Or (Eof (www))
	
				Select replycode$
				          
					Case "200" ' File found...
  					    'Print "found"
		                done:Int=True					
					Case "404" ' File Not found...		
  					    'Print "not found"
 		                done:Int=True										
					Case "301" ' File permanently moved...					
'						station_url:String = location
'                       Print "301 "+location  
 		                done:Int=True
'                    	success:Int=True 
					Case "302" ' File temporarily moved...			
						station_url:String = location
'                       Print "302 "+location  
 		                done:Int=True
                    	success:Int=True 
					Case "303" ' File moved...
'						station_url:String = location
 		                done:Int=True
					Case "307" ' Naughty...                
'						station_url:String = location
 		                done:Int=True					
'                   Default 
'                       done:Int=True
				End Select

	
			EndIf			
			CloseSocket http
			
		EndIf
        done:Int=True

	Until done
		
	Return success
	
End Function


Will test it for stability over the rest of the weekend.

Grisu


Grisu(Posted 2017) [#13]
If someone wants to check it out, I uploaded a build here:
http://www.mediafire.com/file/3twn21a4cle5n1h/prp_ambush.zip
The zip file only includes the Ambush Radio station to keep it small.

|Edit|
Replaced by the full version:
http://pocketradio.awardspace.info/downloads.html


Henri(Posted 2017) [#14]
Glad you got it working. My example was just a proof of consept really. For informational purposes can you give some urls that broke the code ? I'm interested :-)

-Henri


Grisu(Posted 2017) [#15]
One example is GB Radio: http://184.154.202.243:8163

E.g. the "location" string gave back data like it was streaming already.


BlitzSupport(Posted 2017) [#16]

... James code. Less fancy, but it seems to work better. ;)


Woo-hoo! It's definitely not pretty, but seems to work well...

You might want to update the User-Agent string, ie. the name of your client, which is sent to the server:

WriteLine www, "User-Agent: BlitzGet Deluxe"



Grisu(Posted 2017) [#17]
Oh, didn't notice that. Thanks.
Will change it to something else for the next release.