# Automating Search in Yandex

Working with browsers and desktop applications in Sherpa RPA is highly unified.

Let's consider the task of entering text and clicking on the Yandex search bar. While working on this task, you can immediately see several potential problems and learn how to solve them. When recording a click on the "Search" button, pay attention: when the cursor hovers over the button, the button's color becomes slightly darker. This means that at the moment the cursor hovers over the button, its style changes, and thus the selector changes. Additionally, when trying to type or paste something into the Yandex search bar, JavaScript runs in the editing field, which performs instant searches and interferes with the standard way of pasting text into editing fields. To solve this problem, we will use the keyboard emulation function.

Let's write a small script. Click the Record button. Click the button ![](https://sherparpa.ru/wp-content/uploads/2023/11/image57-29w28h-3.png) to emulate keyboard key presses. A window has appeared, which allows you to configure this function.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.1-1.png)

First, you need to use the red crosshair to point to the control element you want to work with.

Note an important point that often occurs when working with websites. As soon as the cursor moves outside the editing field, another rectangle is highlighted with a red border outside the editing field, which is slightly larger than the editing field.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.2-1.png)

The thing is that web pages are also made up of many nested containers, elements, etc., and the problem with multiple levels of nesting on web pages is much more pronounced than in regular applications. Because of this, sometimes you may click in the wrong place and record the wrong action. Often, it is difficult to tell just by appearance which of the overlapping elements is a container and which is a control element. You need to try working with one first, and in case of an error, with the other.

Hover the cursor over the inner rectangle of the editing field. The click has been recorded. Now enter the text for the search.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.3-2.png)

In the window above, you can enter any text in English or Russian. If you need to switch the keyboard layout to enter the next character, the robot will do this automatically. Here, you can also emulate the input of keys or key combinations.

For example, after entering the text, you need to press a key combination, say, CTRL+Shift+S. To do this, double-click on the line where +CTRL is indicated (+ means pressing the button)

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.4-1.png)

Then double-click on the button +Shift

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.5.png)

This means that you pressed Shift, but have not released Ctrl yet, i.e., both keys are pressed together.

Then simply type S on the keyboard.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.6.png)

Next, you need to release the pressed keys. To do this, select the items –Shift and –Ctrl.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.7.png)

Such code, recorded in the text field, will allow emulating the pressing of the key combination CTRL+Shift+S.

Click the Test button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.8.png)

Your query is entered in the Yandex search bar. Click the Save button, and then click the Save and Close button on the recording panel. The result is saved as a block.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.9.png)

Let's consider the properties of the "Press Keys" block.

The "Interactive" property (checkbox) indicates whether the input method will be used, where the entire text is inserted at once, or whether a pause will be emulated after pressing each key. It often happens that when the checkbox is unchecked, scripts running on the page interfere with text input. Therefore, it is better to check the box, although it is faster without it. If the checkbox is checked, you can specify the pause time between key presses in milliseconds.

Select All – a rather useful property. If you check this box, the robot will emulate pressing Ctrl+A before entering text to select all the text already entered in the editing field. This is useful if you are not sure that something has already been typed in the field where you will enter text.

So, after entering our search text, we need to click the Search button. Let's press the record button. Note that when hovering the mouse over the Search button, a situation occurs similar to what was described earlier – there is an inner part of the button, and there is an outer part of the button, and they are nested within each other.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.10.png)

Record a click on the button using the CTRL press.

Review the block that appeared in the designer and its selector. Check the selector's functionality. The selector does not work. After entering text in the search bar, nothing happens, i.e., the Search button is not clicked.

As mentioned earlier, when the cursor hovers over the Search button, the rectangular area of the button becomes slightly darker. This means that there is some script "hanging" on it, which changes the button's selector. This can be easily seen in the selector editing window.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.11.png)

Although editing selectors for browsers is done in the same window as for applications, the appearance of such selectors differs. They usually consist of only two or three lines, with the top line still indicating the process, in this case, Chrome, and the bottom line being the selector we will use to find the desired control element on the page. CSS selector notation is used here, which is not as complicated as it may seem at first glance.

There is a set of either tag names (written in uppercase) or class names (written in lowercase), and they are connected by dots. The dot here means "and." Find the control element that has the tag BUTTON, and in its properties, all the properties listed in the selector are specified:

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.12.png)

In this selector, the last part raises doubts:

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.13.png)

The property hovered\_yes means that the mouse cursor is over the button. It indicates that the button's style changes because the cursor is over it. To correct this selector, you simply need to remove this property from the selector recording along with the dot. The rest can be left.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.14.png)

Check the selector's functionality using the Highlight button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.15.png)

The robot focuses the frame around the desired element.

Check the selector's functionality in practice. In the designer, click the Run button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.16.png)

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.17.png)

The robot executed its script correctly.

Thanks to the fact that the "Select All" checkbox is checked in the properties, the robot first deletes the text that was already in the control element.

So, you have reached the results page, and the next logical action here is to scrape all search results, i.e., you need to get all results in the form of a list – click through all results and get their descriptions. This can be done in at least five different ways.

1. There are blocks of invisible browsers that allow you to download the entire page with results and extract the necessary results from the HTML code. But there are simpler ways.
2. For working with invisible browsers, there is a Yandex search block that implements the task of searching for keywords and obtaining search results from a specified number of pages. It also supports automatic recognition of Yandex Captcha.
3. When visually working with the browser, you can use the "Count Elements" block, which allows you to count how many similar elements are on the page.
4. You can get text using placeholder variables to obtain a certain number of similar elements from the page. But then you need to understand how the CSS selector works to know where to correctly insert the placeholder variables.
5. Another simple way is to use the Extract Data button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.18.png)

This button allows you to extract structured information from websites. Structured information includes lists, tables, and just a set of similar elements on the page. This button can work similarly not only in the browser but also in some applications. With it, you can extract a table from Excel, Word, and even from the 1C interface.

Click the "Extract Data" button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.19.png)

The tooltip window informs you that after clicking the Next button, you need to specify one or two samples of the elements you want to extract. One sample is usually enough to extract data from tables that have been formatted as tables, and two samples are provided for extracting data from search result lists. Click the Next button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.20.png)

Highlight the first result and press CTRL.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.21.png)

The next tooltip window recommends clicking Next in this same window, then hovering the cursor over the SECOND data element (row or cell) to highlight it, and pressing CTRL. Follow this recommendation.

Select the second element on the results page. Note that there are many nested containers here, and you need to select the correct element when clicking.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.22.png)

The window displays all 10 search results extracted from the results page based on the two samples we selected. These results can be automatically saved in a data table.

If you need to extract not only titles from the results list but also, for example, descriptions, addresses, etc., you can use the "Add New Column" button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.23.png)

When you click the button, the system recommends clicking the Next button, hovering the mouse cursor over the sample data element (row or cell) that needs to be added to highlight that element, and then pressing CTRL.

Try to extract domain names.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.24.png)

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.25.png)

A second column has appeared, where the robot collected all the results based on our sample. Note that the results from the first and second columns are matched correctly. That is, the robot understands the structural relationship between the title of each result and the domain name.

If you need to click through the search results and get the URLs of the pages, you can use a special function on the selector editor tab.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.26.png)

Here are the selectors from which the first and second columns were extracted. If something does not work for some reason, you can always manually change these selectors. Additionally, the robot usually provides several different selector options to choose from for a particular element:

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.27-1.png)

There can be several correct options at the same time. When choosing the correct option, it is important to ensure that it does not contain changing parts. In the presented options, there are none. But if you see a strange combination of letters, symbols, and numbers in the selector, it is likely some ID that changes in real-time. That is, every time you visit the page and refresh it, this combination of characters will change. This means that the robot will not be able to find the desired control element at any given time using this selector. Also, a combination of several digits (more than three) is usually suspicious. This is also typically an ID, not a fixed value.

In this case, the selector worked immediately, so we leave the original value for both columns.

If you had changed the selector, you could click the "Update Using New Data" button. Then on the first tab, you would see the updated results. Additionally, you can reselect a column from previously recorded ones or work with the selector list.

To add another column with links, you need to click the corresponding button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.28.png)

The robot determined that the selector "A" would be sufficient, and we would be able to get a link to this element. Click the Update button.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.29.png)

As a result, a new column with the URLs of the pages has been obtained. Click Save.

A pre-configured block "Extract Data" has appeared in the diagram, where everything is ready for extracting results in real-time.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.30.png)

How to check the functionality of this block.

This block sends results to the variable specified in the output property of the result.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.31.png)

You can specify any variable here, but it is customary to first declare the variable that we will work with later. Especially if it is a composite variable.

Declare the variable Table and specify its type – data table.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.32.png)

This is not necessary. If you do not create a variable, but it is used as an output parameter in some property, it will still be created automatically. The convenience of explicitly declaring variables on the variable panel is that you can view the current contents of this table at any step during debugging.

Display the contents of this table on the screen. To do this, use the "Message Window" block from the "Dialog Windows" group.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.33.png)

This block supports outputting both simple and complex data types.

In the properties of the block, select our variable Table. The text of the message displayed on the screen will be taken from it.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.34.png)

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.35.png)

Return to the Yandex page and let's run the entire scenario.

After searching for results based on the query, a window appeared where you can see the contents of all three columns. The data is not separated, but nevertheless, all extracted information is recorded here.

![](https://sherparpa.ru/wp-content/uploads/2024/02/ris.36.png)

Next, you can use the contents of this table in loops, iterating through the rows – using any other tools that the platform offers for working with data tables.