Quantcast
Viewing all articles
Browse latest Browse all 3791

Scraping dynamically generated text from HTMLViewer

I’ve been banging my head against a wall on this seemingly simple task for two weeks now. I’m trying to write a simple Mac app to display a voltage value from a web site. The solar power monitoring web site for my off-grid power system actually hides the battery bank voltage behind a dynamically generated mouseover tip icon that pops up an overlay. I live in the deep Sonoran desert in southern Arizona and during the monsoon season, I live and die by the voltage level of my battery bank when temperatures stay above 110° and the sun is covered by clouds for nearly two months straight.

I’ve figured out how to execute JavaScript in the Desktop HTMLViewer to login to the site automatically and then block just about everything on the home page but what I’m looking for, which results in the final web page displaying in the HTMLViewer just the text I need in an ugly format, but legible.

I figured it would be easy from there to just grab the displayed text from the HTMLViewer and assign the value I need to a variable to be displayed in a TextArea.

I’ve had endless conversations with Perplexity and tried every possible JavaScript option, none of which worked for what I need; innerHTML, outerHTML, innerText, outerText, document.execCommand(‘selectAll’, false, null); document.execCommand(‘copy’);, window.getSelection().toString(), document.getElementsByClassName(), you name it. No success. Even when it does display the HTML source code, the data I need isn’t in it.

In Firefox, I can inspect the code and it’s there (please excuse the rough text grab):

• <i class="tips w" data-ishtml="true" data-bg="#4c483" data-fx="3" data-area="280px" data-tips="3" googl="true"> event
• <table style="width:280px">
+ stbody>
7 <try
<td width="70%">Battery Voltage</td>
♥ <tdy
‹span class="val_i_vBat">50.9</span>
</td>
</tr>
• <tr> (a</tr>
• <tr>@</tr>
• <tr>@</tr>

The class of the voltage value is “val_i_vBat”.

But trying to use JavaScript to grab just that class element has not worked.

I read all the forum entries on the subject I could find, which led me to trying “Cocoa” Selector “copy:” Still, nothing.

Up until now, Xojo has always proven to me that the solution is there, and usually a lot less complicated than I expect, but I’ve made no progress on this from this point.

I assumed it should be as easy as

Var result as string = HTMLViewer1.text
TextArea1.text = result

Not so much.

The original web site looks like this:

After running Javascript in the HTMLViewer to block most everything, it displays this:

All I need is the voltage number after the first line, “Battery Voltage”, which in this case is “50.9V”.

I would describe myself as a beginner to intermediate programmer with Xojo at best. I’ve written a handful of Mac desktop apps to help me with HEVC video compression, video file meta data editing, file renaming and number theory, but this one has resolutely kicked my backside.

8 posts - 3 participants

Read full topic


Viewing all articles
Browse latest Browse all 3791

Trending Articles