TProcess
BlitzMax Forums/BlitzMax Programming/TProcess
| ||
Hi all, I'm developing an application that uses an open source commandline tool but I cannot get Blitzmax to print out all of the output of the program that I would get if I started the same program from the command line. Removing the "If UploadProcess.pipe.readavail()" line causes more output of the program to be printed but also crashes the application. Is there any way around this or should I found an alternative module? Thanks Self.UploadProcess = createprocess("commandline app") Local s:String Repeat If ProcessStatus(UploadProcess)=1 Then If UploadProcess.pipe.readavail() s=UploadProcess.pipe.ReadLine().Trim() print(s) EndIf Else Exit EndIf Forever |
| ||
Not sure where I got this, but this seems to work (Windows example -- uses ipconfig.exe): |
| ||
Is there any way around this or should I found an alternative module? It may be that you're not also reading from the StdErr output, which is another way applications let out more technical output (errors, logs etc.). Right now you're just reading from the StdOut output.I used the following to read a command line application. I found it around these forums; I don't remember who is the author but I'm really grateful to him: |
| ||
Any way for it to process each character one by one, instead of a whole chunk of string text. The application I'm using has a text progress bar and at the moment it just appears to do nothing for a min or two and then shows the completed text progress bar. Also using the err stream as well produces the full results but because of the chucked strings its all out of order. |
| ||
Something which I had to add to the source of the application that I was using was a fflush() call right after it printed something. This fflush() function ensures that the information is output right then. So I sought each printing command inside the source and inserted an fflush() call on the line below them, and rebuilt the software using the build instructions that the author published. The lack of it was giving me a serious synchronization problem, with strings being received by BlitzMax in the wrong order. Apart from that, you can't make the application print one character at a time unless you program it to do exactly that. In a piece of C code the programmer would usually have a statement such as the following to print text to StdOut or StdErr: fprintf( stderr, "Error %d, cannot bind", errno );The entire string is sent out at once. The only way to interpret it character by character is on your end, after you read it in BlitzMax. |
| ||
OK that is fine about the characters but I don't really understand why running this program from command line results in each line printing in the correct order whereas running it with my application results in big chunks of printed strings. This is particularly problematic as my program doesn't receive any err strings until the end, whereas on the cmdline they are inter dispersed between the normal lines. Local s:String Local a:Int = 0 While Process.pipe.readavail() Or Process.err.readavail() If Process.pipe.readavail() Then s=Process.pipe.ReadString (Process.pipe.ReadAvail()) If EmptyString(s) Then Else Console.AppendText(s+" ~n ") EndIf EndIf If Process.err.readavail() Then s=Process.err.ReadString (Process.err.ReadAvail()) If EmptyString(s) Then Else Console.AppendText(s+" ~n ") EndIf EndIf A4SHelperApp.Yield() Wend Commandline output: C:\>arduinouploader standardfirmata\standardfirmata.ino 1 COM 8 Arduino Compiler & Uploader Version 0.8.8 (C)2013 Developed by Stanley Huang <stanleyhuangyc@...;, distributed under GPL license Build Target: Arduino Uno (MCU: atmega328p) Referenced libraries: [Servo] [Wire] [Firmata] Compiling standardfirmata.ino.cpp... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata -I"standardfirmata" "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino .cpp" -o "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino.cpp.o" [Compiliation: 16%] Compiling library [Servo]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Servo/Servo.cpp" -o "C:\Users\Tom\AppData\Local\Temp/ Servo.cpp.o" Compiling library [Wire]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/Wire.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Wi re.cpp.o" avr-gcc -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/utility/twi.c" -o "C:\Users\Tom\AppData\Local\Te mp/twi.c.o" [Compiliation: 33%] Compiling library [Firmata]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Firmata/Firmata.cpp" -o "C:\Users\Tom\AppData\Local\T emp/Firmata.cpp.o" ^C C:\A4S\ArduinoUploader>arduinouploader standardfirmata\standardfirmata.ino 1 COM 8 >>temp.txt Build Target: Arduino Uno (MCU: atmega328p) Referenced libraries: [Servo] [Wire] [Firmata] Compiling standardfirmata.ino.cpp... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata -I"standardfirmata" "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino .cpp" -o "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino.cpp.o" Compiling library [Servo]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Servo/Servo.cpp" -o "C:\Users\Tom\AppData\Local\Temp/ Servo.cpp.o" Compiling library [Wire]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/Wire.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Wi re.cpp.o" avr-gcc -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/utility/twi.c" -o "C:\Users\Tom\AppData\Local\Te mp/twi.c.o" Compiling library [Firmata]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Firmata/Firmata.cpp" -o "C:\Users\Tom\AppData\Local\T emp/Firmata.cpp.o" Linking all objects... Generating program HEX (standardfirmata\standardfirmata.ino.hex)... Compiliation successful completed!Starting upload for Arduino Uno via COM8... C:\A4S\ArduinoUploader>arduinouploader standardfirmata\standardfirmata.ino 1 COM 8 > temp.txt Build Target: Arduino Uno (MCU: atmega328p) Referenced libraries: [Servo] [Wire] [Firmata] Compiling standardfirmata.ino.cpp... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata -I"standardfirmata" "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino .cpp" -o "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino.cpp.o" Compiling library [Servo]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Servo/Servo.cpp" -o "C:\Users\Tom\AppData\Local\Temp/ Servo.cpp.o" Compiling library [Wire]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/Wire.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Wi re.cpp.o" avr-gcc -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/utility/twi.c" -o "C:\Users\Tom\AppData\Local\Te mp/twi.c.o" Compiling library [Firmata]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Firmata/Firmata.cpp" -o "C:\Users\Tom\AppData\Local\T emp/Firmata.cpp.o" Linking all objects... Generating program HEX (standardfirmata\standardfirmata.ino.hex)... Compiliation successful completed!Starting upload for Arduino Uno via COM8... C:\A4S\ArduinoUploader>arduinouploader standardfirmata\standardfirmata.ino 1 COM 8 Arduino Compiler & Uploader Version 0.8.8 (C)2013 Developed by Stanley Huang <stanleyhuangyc@...;, distributed under GPL license Build Target: Arduino Uno (MCU: atmega328p) Referenced libraries: [Servo] [Wire] [Firmata] Compiling standardfirmata.ino.cpp... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata -I"standardfirmata" "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino .cpp" -o "C:\Users\Tom\AppData\Local\Temp/standardfirmata.ino.cpp.o" [Compiliation: 16%] Compiling library [Servo]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Servo/Servo.cpp" -o "C:\Users\Tom\AppData\Local\Temp/ Servo.cpp.o" Compiling library [Wire]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/Wire.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Wi re.cpp.o" avr-gcc -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Wire/utility/twi.c" -o "C:\Users\Tom\AppData\Local\Te mp/twi.c.o" [Compiliation: 33%] Compiling library [Firmata]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cor es/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Serv o -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/ Firmata "arduino/libraries/Firmata/Firmata.cpp" -o "C:\Users\Tom\AppData\Local\T emp/Firmata.cpp.o" Linking all objects... [Compiliation: 50%] Generating program HEX (standardfirmata\standardfirmata.ino.hex)... [Compiliation: 66%] Compiliation successful completed! [Compiliation: 100%] Starting upload for Arduino Uno via COM8... avrdude.exe: AVR device initialized and ready to accept instructions Reading | ################################################## | 100% 0.00s avrdude.exe: Device signature = 0x1e950f avrdude.exe: reading input file "standardfirmata\standardfirmata.ino.hex" avrdude.exe: writing flash (11882 bytes): Writing | ################################################## | 100% 1.90s avrdude.exe: 11882 bytes of flash written avrdude.exe: safemode: Fuses OK avrdude.exe done. Thank you. My codes output: Starting Upload on COM8 Arduino Compiler & Uploader Version 0.8.8 (C)2013 Developed by Stanley Huang <stanleyhuangyc@...;, distributed under GPL license [Compiliation: 16%] [Compiliation: 33%] [Compiliation: 50%] [Compiliation: 66%] [Compiliation: 100%] avrdude.exe: AVR device initialized and ready to accept instructions Reading | ################################################## | 100% 0.00s avrdude.exe: Device signature = 0x1e950f avrdude.exe: reading input file "StandardFirmata\StandardFirmata.ino.hex" avrdude.exe: writing flash (11882 bytes): Writing | ################################################# Build Target: Arduino Uno (MCU: atmega328p) Referenced libraries: [Servo] [Wire] [Firmata] Compiling StandardFirmata.ino.cpp... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cores/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Servo -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/Firmata -I"StandardFirmata" "C:\Users\Tom\AppData\Local\Temp/StandardFirmata.ino.cpp" -o "C:\Users\Tom\AppData\Local\Temp/StandardFirmata.ino.cpp.o" Compiling library [Servo]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cores/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Servo -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/Firmata "arduino/libraries/Servo/Servo.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Servo.cpp.o" Compiling library [Wire]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cores/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Servo -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/Firmata "arduino/libraries/Wire/Wire.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Wire.cpp.o" avr-gcc -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cores/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Servo -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/Firmata "arduino/libraries/Wire/utility/twi.c" -o "C:\Users\Tom\AppData\Local\Temp/twi.c.o" Compiling library [Firmata]... avr-g++ -c -Os -s -pipe -fno-exceptions -ffunction-sections -fdata-sections -MMD -DARDUINO=105 -DF_CPU=16000000L -mmcu=atmega328p -Iarduino/hardware/arduino/cores/arduino -Iarduino/hardware/arduino/variants/standard -Iarduino/libraries/Servo -Iarduino/libraries/Wire -Iarduino/libraries/Wire/utility -Iarduino/libraries/Firmata "arduino/libraries/Firmata/Firmata.cpp" -o "C:\Users\Tom\AppData\Local\Temp/Firmata.cpp.o" Linking all objects... Generating program HEX (StandardFirmata\StandardFirmata.ino.hex)... Compiliation successful completed!Starting upload for Arduino Uno via COM8... |
| ||
Please read my last post again, with emphasis on the fflush() part. It seems to be the same problem. I don't know the exact cause, it must be a difference between MS-DOS and Blitzmax. |
| ||
Thanks for your suggestion but that is a lot of work (also I only really work with blitzmax so compiling a non blitzmax source would be a massive learning curve for me) and doesn't help when I use a program that isn't open source. Surely as it works fine on a command line there must be a workaround for blitzmax? |
| ||
I thought the header line said it's distributed under GPL. That utility is completely open source. In any case, I've done some research and the problem is caused by "output buffering." For efficiency, when the output of an application is piped or redirected, the Input/Output system of the OS buffers the content and releases it only when the buffer is full. This is what causes different streams of content such as StdOut and StdErr to be flushed at different times. MS-DOS is a console, and as such the output is not redirected, so there's no buffering involved and everything is flushed right when printed. - There's no way to request unbuffering from an external application such as your BlitzMax programme. - If you want to print ordered content immediately, without buffering, you need to make changes to the source of the utility that's producing content. This is either adding fflush calls after every print command, or adding a pair of calls to setbuf() at the beginning of the utility source (which is much quicker to do). The calls would be these, added around the beginning of the utility source code: setbuf( stdout, NULL ) setbuf( stderr, NULL ) http://beej.us/guide/bgc/output/html/multipage/setvbuf.html |
| ||
Thanks again for the help :) Yeah the program I'm currently using is open source but I've had this problem in the past with closed source programs and was hoping to fix this problem with those too, oh well. At least I have a workaround for my current project now. |