Web automation question
This is sort of scraping-related...
How can I automate going to a page, copying a particular URL and then sending that URL on through Zapier or Integromat?
-
Specifically, the page to visit would have a different ID In the URL every time, pulled from some input, perhaps in a Zapier/Integromat flow. So, go to https://studio.brightcove.com/products/videocloud/media/videos/XXXXXXXXXX
-
Then I need to copy one of two potential available URLs whose anchor text is "1280 x 720 @ 2114 kbps", and send that URL on through the process (ie. Zapier/Integromat).
I know a bit of Apify, but the login automation is beyond me. No/low code preferred.
I'm interested in Automatio, but it seems browser-based and unconnected to Zapier/Integromat.
What could I use?
Thanks.
Trending on Indie Hackers
Hey @RobertAndrews, I am glad you are aware of Automatio. I just recorded a video for you showing what is Automatio actually and how it can help you in a similar scenario.
Just for the record, Automatio is a cloud/server tool and Chrome Extension is there just to build a bot. But whole automation is happening from the cloud server and has nothing to do with your browser/computer once it's run.
Here you can watch the video https://www.youtube.com/watch?v=fJAYCcbPS5U&feature=youtu.be
Let me know if you have any questions.
Thanks, @kinder, for taking the time with the video.
I signed up for early access a while back, but I just filled in your Google Form.
Here's my workflow in more detail...
I regularly receive videos hosted on Brightcove as assignments to work with. For each video assigned, I need to grab the corresponding raw-video-file URL in a particular resolution, and send it on through a Zapier/Integromat process...
The assigned video IDs (eg.
12345678) typically comes in via a field in Monday.com, a CRM which has Zapier/Integromat connections.The videos have their own individual pages whose URLs include the video ID - so, for example
https://studio.brightcove.com/products/videocloud/media/videos/12345678All of that requires authentication. So, yes, maybe the cookie login method would work, or maybe it requires automated clicking/submission.
Each individual video page like that contains lots of information about the video, including either one or two anchors with text "1280 x 720 @ 2121 kbps" - the raw video file URL...
I want to get one of them - that is the core objective here. I could get them with the Brightcove API but I don't have access to it, so I need to scrape for it behind the authentication/
Then, importantly, I want to send that raw video file URL on to another process, through Zapier or Integromat.
I'm happy to output like Automatio >> Google Sheets >> Zapier/Integromat new-row trigger, if that would work.
However, I wouldn't want to scrape already-scraped URLs, I only want to act on new video IDs. So maybe that sheet would have to be regularly wiped clean so that it only contains new video assignments to act on.
Also, once the bot is set up, I wouldn't want to have to initiate any workflow from within Automatio itself, I wouldn't want to have to actually go into Automatio and click any buttons to start. Instead, whenever those IDs come in from Monday.com, I would just want a workflow to automatically ask Automatio to do what it has to do...
Output: I can probably use the Automatio API with Zapier/Integromat to get the scrape results from Automatio.
Input: However (key question), can Automatio act automatically on a set of URLs fed to it by Google Sheets or an input/POST API request?
I can do similar things in Apify, whether through its API or through its Zapier integration (both to initiate a scrape and to obtain the results), so that Apify just becomes a worker in the chain and I don't have to go there.
Thanks very much.