Form filling and web scraping from a website

84 views Asked by At

Not a good way to start, but pardon me if this if off-topic, it seems like a programming question though...

From an ASP.NET website I want to open a page in a new browser that has a toolbar at top and an iframe-like window at the bottom. The frame-like window will support tabbed browsing and load a third party website. The toolbar will have buttons that allow the user to manipulate the HTML (form-fill and web-scrape). For example, toolbar buttons may be "Extract Webpage Data" or "Fill Form".

Ideally it would work with IE, Edge, Chrome and Safari, but an absolute minimum requirement is IE, a more preferable minimum requirement is Chrome and Edge.

I have seen this done, well, by other proprietary software. I do not know if they require a specific browser (like IE where they can install a plugin) or how they do it, that is my question.

So I have narrowed this problem down to three possibilities:

  1. Use pure HTML, Javascript, et al. - Using an iFrame almost works perfectly but the content will not be in the same domain so I cannot access the iFrame's HTML.

  2. Use (or write) a proprietary browser - I do not think you can (or want) to launch an EXE from a web page, plus this seems rather complex in itself.

  3. Use (or write) a plug-in - Probably limits use to IE. I think an IE plugin could do what I want based on other plugins I've seen.

I have past desktop programming experience with a web automation and scripting product, while promising, I don't think they offer what I need:

They have an ASP.NET COM component that runs server side so it does not display an interface to the user but can be used to silently fill and scrape a website based on scripts.

They also have a proprietary browser that shows a user interface and runs scripts to fill and scrape. But this is an EXE, so cannot be launched from a web site.

They have an IE Plugin, that adds a companion popup window that attaches itself to IE. Similar to their browser and runs scripts.

Question - This can be done, I've seen it, but what is the mechanism? I'm leaning to an IE plugin.

If plugins are the answer, chrome has extensions, is that a possibility?

0

There are 0 answers