Loading images from the web?

BlitzMax Forums/BlitzMax Beginners Area/Loading images from the web?

Booticus

(Posted 2005) [#1]

Hey all. Using the stream command to load data from a webpage, Blitz Example:

in = ReadStream("http::blitzbasic.com")

If Not in RuntimeError "Failed to open stream"

While Not Eof(in)
	Print ReadLine(in)
Wend

CloseStream in

That displays all the HTML code, which is neat, but is there any way to load an image that you know will always be there:

Example: (didnt want the real link, its a big picture!)
(add the http)://goes.gsfc.nasa.gov/goeswest/pacific/color/0000_latest.jpg

That will theoretically always be there, waiting for you to grab it. I thought it would be kind of cool to grab snapshots of satellite images every few hours, make an animation, save for posterity, etc. Any pointers would be appreciated!

Yan	(Posted 2005) [#2]

I was possessed by the spirit of Skidracer...According to the Wiki, at least... ;o)

Graphics 400, 300, 0

DrawText "Getting Image...", 140, 144
Flip 

img:TImage = LoadImage(LoadBank("http::goes.gsfc.nasa.gov/goeseast/hurricane2/color_lrg/latest.jpg")) 
If Not img Then RuntimeError "Unable to download image"

Graphics img.width, img.height, 0 

DrawImage img, 0, 0
Flip

WaitKey()

End

DannyD

(Posted 2005) [#3]

Nice !

Booticus

(Posted 2005) [#4]

Thanks to both of you!!! Thats EXACTLY what I was lookin for!!! PERFECT!!! Now just where the heck did Skid share this??!?! I was searching the forums. Maybe I didn't word it right? Just for future reference of course!!!

Booticus

(Posted 2005) [#5]

Um....now anyone have a way of SAVING jpgs from a TImage? I saw xacto had a little beta of some work he was doing:

http://www.blitzmax.com/Community/posts.php?topic=41965

cept now its outdated and doesn't work, somn about a fontglyph error. Course that was made back in December. Anyone? :) I wont push my luck, I can still save out as PNG, but they're pretty giant compared to JPGs. :) I already searched the forums with no luck.

Yan	(Posted 2005) [#6]

Yan	(Posted 2005) [#7]

I'll probably find a use for this at some point, so I had a quick fiddle with Jeffrey's code, and a re-write of 'JPGLoader.bmx' later...

Overwrite 'mod/brl.mod/JPGLoader.mod/JPGLoader.bmx' with...

Strict

Rem
bbdoc: JPG loader
end rem
Module BRL.JPGLoader

ModuleInfo "Version: 1.02"
ModuleInfo "Author: Simon Armstrong, Jeffrey D. Panici"
ModuleInfo "License: Blitz Shared Source Code"
ModuleInfo "Copyright: Blitz Research Ltd"
ModuleInfo "Modserver: BRL"
ModuleInfo "History: 1.02 Release"
ModuleInfo "History: Added support for monochrome / single channel"

Import BRL.Pixmap
Import Pub.LibJPEG

Private

Global jpg_stream:TStream

Function readfunc%( buf@Ptr,size,nmemb,src:Object )
	Local n
	n=TStream(src).ReadBytes( buf,size*nmemb )
	Return n/size
End Function

Function writefunc%( buf@Ptr, size, nmemb, src:Object )
	Local n
	' N.B. Not sure why this is happening but LibJPEG
	'      is passing the wrong size in nmemb
	'      from jpeg_finish_compress.  We adjust it
	'      here but this isn't very pretty.  I'll need
	'      to trace through jpeg_finish_compress to see
	'      why this is happening.
	If nmemb < 4096 Then nmemb:-1
	
	n = TStream( src ).WriteBytes( buf, size * nmemb  )
	
	Return n / size
End Function

Public

Rem
bbdoc: Load a Pixmap in JPG format
about:
#LoadPixmapJPG loads a pixmap from @url in JPG format.<br>
<br>
If the pixmap cannot be loaded, Null is returned.
End Rem
Function LoadPixmapJPG:TPixmap(url:Object)
	Local	jpg,width,height,depth,y
	Local	pix:Byte Ptr	
	Local	pixmap:TPixmap
	jpg_stream = ReadStream(url)
	If Not jpg_stream Then Return

	If loadjpg(jpg_stream,readfunc,width,height,depth,pix) Or (width = 0)
		jpg_stream.Close()
		jpg_stream = Null
		Return
	EndIf
		
	Select depth
	Case 1
		pixmap=CreatePixmap( width,height,PF_I8 )
		For y=0 Until height
			CopyPixels pix+y*width,pixmap.PixelPtr(0,y),PF_I8,width
		Next
	Case 3
		pixmap=CreatePixmap( width,height,PF_RGB888 )
		For y=0 Until height
			CopyPixels pix+y*width*3,pixmap.PixelPtr(0,y),PF_RGB888,width
		Next
	End Select
	
	jpg_stream.Close()
	jpg_stream = Null
	free_ pix			
	
	Return pixmap
End Function

Rem
bbdoc: Save a Pixmap in JPG format
about:
#SavePixmapJPG saves @pixmap to @url in JPG format. If successful, #SavePixmapJPG returns
True, otherwise False.<br>
<br>
The optional @quality parameter should be in the range 1 to 100 (defaults to 50), where
1 indicates lowest quality (smallest file size) and 100 indicates highest quality (largest file size).
End Rem
Function SavePixmapJPG:Int( pixmap:TPixmap, url:Object, quality=50 )
	Assert (quality >= 1) And (quality <= 100), "quality value out of range"
	
	jpg_stream = WriteStream( url )
	If Not jpg_stream Then Return
	
	Local result = savejpg(jpg_stream, writefunc, pixmap.width, pixmap.height, pixmap.PixelPtr( 0, 0 ), quality)
	jpg_stream.Close()
	jpg_stream = Null
		
	Return result
End Function

Private

Type TPixmapLoaderJPG Extends TPixmapLoader
	Method LoadPixmap:TPixmap( stream:TStream )
		Return LoadPixmapJPG(stream)
	End Method
End Type

New TPixmapLoaderJPG

Overwrite 'mod/pub.mod/libjpeg.mod/libjpeg.bmx' with...

Strict

Module Pub.LibJPEG

ModuleInfo "Version: 1.03"
ModuleInfo "Author: Independent JPEG Group"
ModuleInfo "License: Freely distributable"
ModuleInfo "Copyright: Independent JPEG Group"
ModuleInfo "Modserver: BRL"
ModuleInfo "Credit: Adapted for BlitzMax by Simon Armstrong"

ModuleInfo "History: 1.03 Release"
ModuleInfo "History: 1.02 Release"
ModuleInfo "History: Fixed C Compiler warnings"

Import "jcapimin.c" 
Import "jcapistd.c" 
Import "jccoefct.c" 
Import "jccolor.c" 
Import "jcdctmgr.c" 
Import "jchuff.c" 
Import "jcinit.c" 
Import "jcmainct.c" 
Import "jcmarker.c" 
Import "jcmaster.c" 
Import "jcomapi.c" 
Import "jcparam.c" 
Import "jcphuff.c" 
Import "jcprepct.c" 
Import "jcsample.c" 
Import "jctrans.c" 
Import "jdapimin.c" 
Import "jdapistd.c" 
Import "jdatadst.c" 
Import "jdatasrc.c" 
Import "jdcoefct.c" 
Import "jdcolor.c" 
Import "jddctmgr.c" 
Import "jdhuff.c" 
Import "jdinput.c" 
Import "jdmainct.c" 
Import "jdmarker.c" 
Import "jdmaster.c" 
Import "jdmerge.c" 
Import "jdphuff.c" 
Import "jdpostct.c" 
Import "jdsample.c" 
Import "jdtrans.c" 
Import "jerror.c" 
Import "jfdctflt.c" 
Import "jfdctfst.c" 
Import "jfdctint.c" 
Import "jidctflt.c" 
Import "jidctfst.c" 
Import "jidctint.c" 
Import "jidctred.c" 
Import "jmemmgr.c" 
Import "jmemnobs.c" 
Import "jquant1.c" 
Import "jquant2.c" 
Import "jutils.c" 
Import "loadjpeg.c"

Extern

Function loadjpg(stream:Object,reado(buf@Ptr,size,nmemb,src:Object),width Var,height Var,depth Var,pix:Byte Ptr Var)
Function savejpg(stream:Object, writeo(buf@Ptr, size, nmemb, src:Object), width, height, pix:Byte Ptr, quality)

End Extern

Overwrite 'mod/pub.mod/libjpeg.mod/libjpeg.c' with...

// loadjpeg.c

// jpeg wrapper for BlitzMax libjpeg

#include <stdio.h>
#include <setjmp.h>
#include <jpeglib.h>

static jmp_buf jmp_env;

static void format_message (j_common_ptr cinfo, char * buffer) {}
static void output_message (j_common_ptr cinfo) {printf("jpeg message\n");}
static void emit_message (j_common_ptr cinfo, int msg_level) {}
static void error_exit (j_common_ptr cinfo) { longjmp( jmp_env,-1 ); }//printf("jpeg error_exit\n");}//jpeg_destroy(cinfo);}
static void reset_error_mgr (j_common_ptr cinfo) {cinfo->err->num_warnings=0;cinfo->err->msg_code = 0;}

int (*ReadStream)(void*,int,int,void*);
int (*WriteStream)(void*,int,int,void*);

void initjerr(struct jpeg_error_mgr *jerr)
{
	jerr->error_exit = error_exit;
	jerr->emit_message = emit_message;
	jerr->output_message = output_message;
	jerr->format_message = format_message;
	jerr->reset_error_mgr = reset_error_mgr;
	jerr->trace_level = 0;		// default = no tracing 
	jerr->num_warnings = 0;	// no warnings emitted yet 
	jerr->msg_code = 0;		// may be useful as a flag for "no error" 
// Initialize message table pointers 
	jerr->jpeg_message_table = NULL;		//jpeg_std_message_table;
	jerr->last_jpeg_message = 0;		//(int) JMSG_LASTMSGCODE - 1;
//  jerr->jpeg_message_table = jpeg_std_message_table;
//  jerr->last_jpeg_message = (int) JMSG_LASTMSGCODE - 1;
	jerr->addon_message_table = NULL;
	jerr->first_addon_message = 0;	// for safety 
	jerr->last_addon_message = 0;
}

int loadjpg(void *stream,void *readfunc,int *width,int *height,int *channels,char **pix)
{
	int		size,w,h,d,span,res;
	char	*p;
	
	struct jpeg_decompress_struct cinfo;
	struct jpeg_error_mgr jerr;
	
	ReadStream=readfunc;
		
	initjerr(&jerr);
	cinfo.err=&jerr;	
	
	jpeg_create_decompress(&cinfo);
	jpeg_stdio_src(&cinfo,(FILE*)stream);
	
	if( setjmp(jmp_env) ){
		return -1;
	}
	
	res=jpeg_read_header(&cinfo,TRUE);
	if (res!=1) return -1;

	jpeg_start_decompress(&cinfo);
	
	*width=w=cinfo.output_width;
	*height=h=cinfo.output_height;
	*channels=d=cinfo.output_components;
	p=(char*)malloc(w*h*d);
	*pix=p;
	if (p)
	{
		span=w*d;
		while (h--)
		{
			jpeg_read_scanlines(&cinfo,(JSAMPARRAY)&p,1);
			p+=span;
		}
	}
	jpeg_finish_decompress(&cinfo);	
	jpeg_destroy_decompress(&cinfo);
	return 0;
}

int savejpg( void* stream, void* writefunc, int width, int height, char* pix, int qlty ) {
	struct jpeg_compress_struct	cinfo;
	struct jpeg_error_mgr jerr;
	int span;

	WriteStream = writefunc;
	initjerr( &jerr );
	cinfo.err = &jerr;	
	jpeg_create_compress( &cinfo );
	jpeg_stdio_dest( &cinfo, (FILE*)stream );
	if( setjmp( jmp_env ) ) {
		return 1;
	}
	cinfo.image_width = width;
	cinfo.image_height = height;
	cinfo.input_components = 3;
	cinfo.in_color_space = JCS_RGB;
	jpeg_set_defaults( &cinfo );
	jpeg_set_quality( &cinfo, qlty, TRUE );
	jpeg_start_compress( &cinfo, TRUE );
	span = width * cinfo.input_components;
	while( cinfo.next_scanline < cinfo.image_height ) {
		jpeg_write_scanlines( &cinfo, (JSAMPARRAY)&pix, 1 );
		pix += span;
	}
	jpeg_finish_compress( &cinfo );
	jpeg_destroy_compress( &cinfo );
	return 0;
}

/*
int readjpg(canvas *m,input *in,int flags)
{
	jpeg_decompress_struct	cinfo;
	jpeg_error_mgr			jerr;
	u8						*buffer,*b;
	int						span,chan,x,y,isjpg;
	char					hdr[10];

	int n=in->read(hdr,10);
	isjpg=false;
	if (hdr[6]=='J'&&hdr[7]=='F'&&hdr[8]=='I'&&hdr[9]=='F') isjpg=true;
	if (hdr[6]=='E'&&hdr[7]=='x'&&hdr[8]=='i'&&hdr[9]=='f') isjpg=true;
	if (!isjpg) return 0;
	in->skip(-10);
// init jpeg lib
	initjerr(&jerr);
	cinfo.err=&jerr;	
	jpeg_create_decompress(&cinfo);
	jpeg_stdio_src(&cinfo,(_iobuf*)in);
	jpeg_read_header(&cinfo,TRUE);
	jpeg_start_decompress(&cinfo);
// init rmap
	m->resize(cinfo.output_width,cinfo.output_height,flags,false);
	chan=cinfo.output_components;
	span=m->width*chan;
	buffer=new u8[span];
// read
	nitro->lock(m,canvas::WRITELOCK);
	for (y=0;y<m->height;y++)
	{
		jpeg_read_scanlines(&cinfo,(JSAMPARRAY)&buffer,1);
		b=buffer;
		for (x=0;x<m->width;x++)
		{
			m->writepixel(x,y,255,b[0],b[1],b[2]);
			b+=chan;
		}
	}
	nitro->release();
// close
	jpeg_finish_decompress(&cinfo);	
	delete buffer;
// finit
	jpeg_destroy_decompress(&cinfo);
	return true;
}

void writejpg(canvas *can,output *out,int qlty)
{
	jpeg_compress_struct	cinfo;
	jpeg_error_mgr			jerr;
	u8						*buffer,**bptr;
	int						y,w,h;

	initjerr(&jerr);
	cinfo.err=&jerr;	
	jpeg_create_compress(&cinfo);
	jpeg_stdio_dest(&cinfo,(_iobuf*)out);
	cinfo.image_width=can->width;			//* image width and height, in pixels
	cinfo.image_height=can->height;
	cinfo.input_components=3;				// # of color components per pixel
	cinfo.in_color_space=JCS_RGB;			// colorspace of input image
	jpeg_set_defaults(&cinfo);
	jpeg_set_quality(&cinfo,qlty,TRUE);
	jpeg_start_compress(&cinfo,TRUE);
	w=can->width;
	h=can->height;
	buffer=new u8[w*3];
	bptr=&buffer;
	nitro->lock(can,canvas::READLOCK);
	for (y=0;y<h;y++)
	{
		can->readpixels(0,y,w,1,buffer,canvas::RGB888);
		jpeg_write_scanlines(&cinfo,bptr,1);
	}
	nitro->release();
	jpeg_finish_compress(&cinfo);
	jpeg_destroy_compress(&cinfo);
}

*/


//	struct jpeg_error_mgr jerr;
	//struct j_common_ptr cinfo;

From the IDE, do a 'Program>Build Modules', and you should be good to go after a restart.

You may also want to...

<BMAX PATH>/bin/bmk docmods
<BMAX PATH>/bin/bmk syncdocs

...as I don't think 'Build Mods' does this.

Booticus

(Posted 2005) [#8]

Thanks! Thats totally cool!

xlsior

(Posted 2005) [#9]

Any chance the SaveJPG can be included in the official build?

Robert Cummings

(Posted 2005) [#10]

hang on, how do you save a file you stream in, with identical data?

for example if he's streaming in a jpeg, surely it is already a jpeg?

Yan	(Posted 2005) [#11]

Yup!...You could just save the bank...

jpgBnk:TBank = LoadBank("http::goes.gsfc.nasa.gov/goeswest/pacific/color/0000_latest.jpg")
If Not jpgBnk Then RuntimeError "Unable to download image"

'img:TImage = LoadImage(jpgBnk)
'...ETC...

SaveBank(jpgBnk, "c:/test image.jpg")

End

But as I said, I'd probably be needing the ability to save JPGs anyway and Jeffrey had already done 99.9% of the work. Also, Booticus might want to manipulate the image in some way before saving.

Booticus

(Posted 2005) [#12]

Hell yeah to both of ya!!! These are perfect bits of code for me and anyone else interested in!!!

Now One eyed jack, are you releated to Two eyed Pete?? The names and all. ;)

Grisu

(Posted 2006) [#13]

Can someone please make Jeremy's code to work with bmx 1.18.

I only need a "SavePixmapJPG" function.

skidracer

(Posted 2006) [#14]

SavePixmapJPeg will feature in the next update.

Grisu

(Posted 2006) [#15]

I guess I can wait that long! :)

Thanks for letting us know of this!!!!!!!!!!!!!

patmaba

(Posted 2007) [#16]

when i execute your code :

in=ReadStream("http::blitzbasic.com")

If Not in RuntimeError "Failed to open a ReadStream to file"

While Not Eof(in)
Print ReadLine(in)
Wend
CloseStream in

I receive the following error. what is bad ?

BlitzMax Debug Report:
Unhandled Exception:Failed to open a ReadStream to file

End of Debug Report.

tonyg

(Posted 2007) [#17]

The program is trying to connect to the internet. If you have a firewall then you need to allow access for this program. If you have a firewall and don't get a 'allow/deny' pop-up then you're being prevented by your firewall policy.

patmaba

(Posted 2007) [#18]

I'm using a proxy.

How to set the source to bypass proxy ?

Dabz	(Posted 2007) [#19]

@Yan: thats some juicy bit code you have given us there,big thank you from me