download an image using inet1.execute instead of .open

[David]

Hello VB ppl,

I'm writing a custom scraper tool that will fetch web data from a specific site. Normally I would use the open method with internet transfer control but this specific site rejects http requests from the user agent associated with open. If I use the execute method and custom user agent string I can get in. The tool works fine for the text of the page, but now I want to tweak it to fetch images as well. I'm having trouble figuring out how to populate my binary data variable to take in the stream produced by the control. I've searched this group and other resources but I'm just more confused now. Maybe someone can help me out.

The basic fetch code:

sub getImage()
Inet1.Protocol = icHTTP
Inet1.Execute URL, "GET", , "User-Agent: testing" & vbCrLf
While Inet1.StillExecuting
DoEvents
Wend
end sub

Now if this was still text data I wanted I would use this:

Sub Inet1_StateChanged(ByVal state As Integer)
Dim data$, finalData$
Select Case state
Case 12 'icResponseCompleted
data = Inet1.getChunk(4096)
While data <> ""
finalData = finalData & data
data = Inet1.getChunk(4096)
Wend
'file write routine using finalData
...
end sub

But with binary data I'm getting type mismatch errors and such. For one thing I don't know how to test whether the stream is done or not (as done above by testing if data=""). Nor do I know how to concatenate two binary variables or what the equivalent process would be if concatenate is the wrong concept.

Sub Inet1_StateChanged(ByVal state As Integer)
Dim data() As Byte, finalData() As Byte
Select Case state
Case 12 'icResponseCompleted
data = Inet1.getChunk(1024, icByteArray)
While data > 0
finalData = finalData & data
data = Inet1.getChunk(1024, icByteArray)
Wend
'file write routine using data().
...
End Select
End Sub


Any ideas or suggestions? I think once I get the data into the variable properly I can figure out how to write it out as a file. Thanks for any assistance.

--David

[Deanna Earley]

Post by David
Hello VB ppl,
I'm writing a custom scraper tool that will fetch web data from a
specific site. Normally I would use the open method with internet
transfer control but this specific site rejects http requests from
the user agent associated with open. If I use the execute method and
custom user agent string I can get in. The tool works fine for the
text of the page, but now I want to tweak it to fetch images as well.
I'm having trouble figuring out how to populate my binary data
variable to take in the stream produced by the control. I've
searched this group and other resources but I'm just more confused
now. Maybe someone can help me out.
sub getImage()
Inet1.Protocol = icHTTP
Inet1.Execute URL, "GET", , "User-Agent: testing" & vbCrLf
While Inet1.StillExecuting
DoEvents
Wend
end sub

<SNIP>

Post by David
But with binary data I'm getting type mismatch errors and such. For
one thing I don't know how to test whether the stream is done or not
(as done above by testing if data=""). Nor do I know how to
concatenate two binary variables or what the equivalent process would
be if concatenate is the wrong concept.

<SNIP>

Post by David
Any ideas or suggestions? I think once I get the data into the
variable properly I can figure out how to write it out as a file.
Thanks for any assistance.

VB6 doesn't allow you to just append to an array.

Your best bet is to create a large byte array (to fit the image you
expect), and then CopyMemory() each chunk into that point of your final
buffer.
You can check the size to see if the new buffer will fit first then
extend it (Redim Preserve) by a reasonable amount if needed.
When it's finished, you know how much you've received and then truncate
the array (Redim Preserve again).

[Theo Tress]

Post by Deanna Earley
VB6 doesn't allow you to just append to an array.

But having an array A(100) you can do a ReDim Preserve A(300) to obtain a
larger array and specify A(100) as base address in a subroutine call instead
of A() which would implicitly specify A(0), and in that way you can import
dara into an array beginning at position 100 in an easy way.

[Deanna Earley]

Post by Theo Tress

Post by Deanna Earley
VB6 doesn't allow you to just append to an array.

But having an array A(100) you can do a ReDim Preserve A(300) to obtain
a larger array and specify A(100) as base address in a subroutine call

Correct.

Post by Theo Tress
instead of A() which would implicitly specify A(0)

Incorrect.

A or A() passes the array itself.
A(0) passes the first item (whatever type it is)
A(100) Passes the 101th item (whatever type it is)

IF you're calling a Win32 API function (like CopyMemory) that expects a
pointer, you can pass VarPtr(a(XX)) (or a(XX) by ref) and they can work
out the rest from there, but anything that expects or returns "an array"
needs the full array, unless it also accepts an offset as a separate
parameter.