# Picture-in-Picture Mode (PiP)

**PiP Mode** is a special mode for connecting to your own computer via the RDP protocol, allowing you to work in a mini-window simultaneously with the main session window. For example, the User can interact with the web interface while the Robot performs its Tasks in another window.

## 1. Key Features of PiP Mode

**Multitasking**: simultaneous interaction with multiple applications (possible in cases where the application itself supports it; many programs do not allow running multiple instances at the same time).

**Increased Efficiency**: The Robot is capable of interacting with applications in a way that does not interfere with the User.

PiP mode allows simultaneous interaction with the interfaces of both the Robot and the User on one screen. This means that:

* The Robot can perform its tasks in the background in a separate window.
* The User can simultaneously manage the interface, input data, click on elements on the screen, or view information.

Suppose the User has automated scripts running and needs to check or change something manually. Thanks to PiP mode, everything happens on one screen with the ability to switch or overlap.

* The active interface is the interface currently being controlled. For example, the User clicks the mouse on the active window.
* Automated actions refer to the interface where actions can be performed by the Robot either in the background or in parallel with manual control.
* In PiP mode, system mouse control is often switched or simulated for specified windows.
* When clicking or moving the mouse, the User directly controls the current active window, while the Robot, if running, can work in another window.
* In some scenarios, mouse locking is used.

> In PiP mode, you can configure automatic switching of main windows or operate in "synchronous" mode — this depends on the scenario.

## 2. Configuring the Robot for PiP Mode

To work correctly in PiP mode, the following steps must be completed:

#### 2.1. Configuring Group Policy

* Open the Local Group Policy Editor.
* Navigate to: `Computer Configuration` → `Administrative Templates` → `System` → `Credential Delegation`.
* Enable the setting: Allow delegating default credentials.
* Ensure it is set to Enabled.

#### 2.2. Configuring User Rights

The User needs the following rights:

* `Allow Log On Locally`
* `Access this computer from the network`

These policies can be set through `Computer Settings` → `Windows Settings` → `Security Settings` → `Local Policies` → `User Rights Assignment`.

> Administrator rights are required to activate PiP mode. Once activated, any User will be able to use PiP mode without needing administrator rights.

## 3. Starting the Robot in PiP Mode

The Robot in PiP mode can be started in two ways:

#### 3.1. Through the Sherpa Designer Interface:

The User can start in PiP mode through the "Run Robot" block in Sherpa Designer. To do this, they should:

1. On the "Debug" panel, click the button ![](https://lh7-rt.googleusercontent.com/docsz/AD_4nXch9OxukQotZGwwsSJOlq5hithPHU8nNjlCBq39vw41ZFB7hT8Ng0WQ3nl6vlnvpDKl0k96g6DXZFDMpHAlioNZL52aL1S7nDaqjgvPP3-LB1MNCjFRr-p9skrs1oZxq6Y5C00Qqg?key=VkzMAJy7plKoC0RBMeOXBw).

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfagfT5utUEr9o45RCt82tqdldfc16RzQ4kjUUUiM-fXBnELs9vUf13n9EQRPtRhOjuPqPBFNyN14iujR5jt49EJaPuwNrxBJoAtEy3G8IY_Md-Yj1A3jI7eDrUYL0f4Lp3UzvYvg?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

2. Click on "Run in PiP Mode (Picture-in-Picture)":

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXcvcho68UGHz5YjSeBmlVBJC6F_lX9vwWJ4Z_BnALTGkuyeMx4uZ4d2aYR9zaYLdaVMP6N_1GIFKrMzWAfwk9i8h1Ow_03NE4LCuBhVTjJfOFToBSGwJ55oZwumm-1vEWpdQgn1BA?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

3. Confirm the action by clicking the “Yes” button:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfSLgTYXPTwXtB5UgpHilS2xp5J51efYJQnq6ctYy-wWX5QkiS52pNRrBxw-rwqoKmKS39cYo9CnQwCBqxmLcvH5tqOnyvm9qJr4VBW01TT9rC4F7cPsJh8EEfsqTHI_ek05UWOyw?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

> For correct operation with browsers, ensure that the browser is not running in the main session; otherwise, the Robot will not be able to interact with it.

#### 3.2. Through the Sherpa Assistant Interface:

The User can start in PiP mode through Sherpa Assistant by doing the following:

1. In the "Robots" window, select the desired Robot and click the button ![](https://lh7-rt.googleusercontent.com/docsz/AD_4nXehdddPEIIjITQ62I_Ts3xKFVd721sHz8ZYtdyTM3fwBk8d_6wUFnyqbppFBJUU7RinhhAkd_gvcaS0pXDsMtisBNpTovHX0zFSpNVK_HOuozC0BYi2vwzgcA1yrsCKCFYAtL_E-Q?key=VkzMAJy7plKoC0RBMeOXBw):

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXdI04dHhp2avVAHqjh9e6wZdcBz9qyCA4tKZill8TTUIAbxjPlNe7HB4EbMNA2E8gmAtq3Tm9JFbB2inGw215OkM1phQChxVEQUgARUOLJfMT4kxOWjitqEN9wTjHS8JiKWatv5kw?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

PiP client launch settings:

* **“Always run in PiP client mode”** – when this option is enabled, the Robot will always launch with the PiP client interface visible to the User.
* **“Always run PiP client locked”** – the Robot starts in a locked state (the PiP client interface is open, but the User cannot influence the Robot's operation).
* **“Always run PiP client in hidden mode”** – the Robot starts, but the PiP client window is hidden and not displayed on the screen.

2. Enable the desired mode and set hotkeys:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXdbehARw7y7mBW6p8dyLkVPlcbETJOQwEqYwVWvntc9VK7p6q9sHKQ4ronx3LQVeDgZ9_Pt0Amp7EUrR8DAWJ0vR47pImdWXb0toUjWyhQvb1axd6fe3dRMWV70K1HpYgzUHqwZsQ?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

To access Processes synchronized with the Orchestrator, authorization through the User is required.

To do this:

* In the Sherpa Assistant tray menu, select the “Robots” item:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXdQuzhtgrMDzf4FWQ9qvWxL8L0jD2_S7O5QrDL5g9mtjUGWIsx47Izq6gY5gaxULrb-BNK2aVYiThxHj77UCu8CWgHglIRDzB8Y1lvmlPZrovrmhVCGlmdOoB4_mwqyWRrYOZbPlQ?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

* In the upper part of the window, click the “Settings” button:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXf3kYrW5BTzs9WNzRYUzVv9SVaD1Trw0ruQeyM13QSjE6IiFxuk6fXQADewwWXddobWCU3fUEZL1g1yRxm_jSexj5oWmo30vFcji8cVeae4WoWaA6SEE8VJDu5GMhefxNiRcxNaoQ?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

* In the pop-up “Settings” window, enter the login and password:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXfOfxvo4l1Y_C_59TqOOd3nXmlp5oU0u5NSiNCzfs1iOlVCA77p7GAcXu6YNrvv3OaMgLPgiXRUt1RnuFhRiS-FkiWZ4Axt8JkUCzWKLRUpH5sqJSS4zcOebPvTNQYmQoIc-XWB?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

* Click the “OK” button.

After logging in, the User will have access to Processes synchronized with the Orchestrator:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXeJ3I7DRNZmAIYkiYeCTytZQJuBU1Ji9VgZAdDgXeBqHsVYdfxrMzylrspHGfKHo5cCP9zdyVfiUTvcV-uCdvDGpjrD5XIi1Izlxa1Hu_IAuv1rA-cl9VmEqC1bnRJ7819GIAC_Jw?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

They can also be seen on the [Processes](https://docs.sherparpa.ru/sherpa-orchestrator/rabota-v-sherpa-orchestrator/ekrany/processy) screen:

<figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXf9dTwpDq_fWtkvGV5rge4s80XIykagAxoCoIRSmmbvkxSr48A3ymtYNQdlDThHjqPSNGEfVjDwAbMMHs2xnREFCDwHuQXNr49o0VoVTklaTECrxKWkok5hBJjfdZ6oBw2W6EM2qw?key=VkzMAJy7plKoC0RBMeOXBw" alt=""><figcaption></figcaption></figure>

#### 3.3. Important Recommendations and Limitations

1. When working in normal mode, the User can interfere with the Robot's Work, especially if several tabs and/or applications are open on the screen at the same time. To address this issue, the Mouse Lock feature is configured in PiP mode.

Mouse Lock prevents accidental mouse clicks during the Robot's Work. However, it should be noted that such a feature may create difficulties for the Robot itself, for example, if interaction with the interface is required. In such cases, the Robot will not be able to perform all necessary actions.

2. If the User encounters issues with selectors when launching a Java application in PiP mode, it is recommended to check the resolution of the opening window in the Robot Log at the path:

`c:\Users\User\AppData\Roaming\Sherpa RPA Data\!PiPClient\Log\`

3. If Outlook is open on the main desktop, the Robot running in PiP mode may encounter an error. It is not possible to work with two instances of Outlook simultaneously. The User needs to choose where it is more convenient to work: in PiP or on the main desktop.

Current limitations arise due to the operation of the Outlook COM object, which can only connect to the first instance of the application. If the User's system uses Exchange, direct interaction with it can be attempted.

#### 3.4. Automating Applications in Invisible Mode

Automating applications such as Power BI can be effectively implemented in invisible mode using PiP technology.

The implementation of PiP mode is carried out through User settings, providing the ability to manage automation more conveniently and efficiently. For more detailed information and step-by-step instructions on configuration, you can refer to the [Microsoft documentation](https://learn.microsoft.com/ru-ru/windows/win32/termserv/child-sessions).

## 4. Use Cases

#### 4.1. Example 1: Automated Script with Clicks + Manual Input

* The Robot performs a series of clicks on the interface (for example, on buttons in a web page).
* The User inputs new data or checks the result while manually managing the interface.

> **Implementation**
>
> The Robot is launched in PiP mode with active mouse control in a specific area, while the User can, for example, click on other elements or input text.

#### 4.2. Example 2: Testing and Automating the Interface

* The Robot automatically tests the program's interface.
* While the script is running, the User (tester) observes and intervenes if necessary (for example, stops the script or corrects the script).

> **Implementation**
>
> Implementing monitoring and pause:
>
> * Enabling monitoring mode — the script is demonstrated in the PiP window.
> * Implementing "Pause", "Stop", "Continue" buttons if the script is waiting for confirmation or manual intervention.
> * Implementing automatic checks (for example, searching for text or images to confirm successful completion of stages).
>
> Error handling and incorrect situations:
>
> * Adding checks to the script (for example, for the presence of an element, for successful completion of an action).
> * Adding a notification of stopping in case of an error, or configuring automatic retry attempts.
>
> Completing the test and collecting reports:
>
> Outputting logs, screenshots, or reports.
>
> Collecting statistics.

## 5. Switching Control

In PiP mode, you can configure the system so that the Robot performs actions in parallel with the User, or conversely, switch control (for example, so that the Robot works until the User forcibly stops it).

#### 5.1. Parallel Work of the Robot and User

In this case, the Robot and User work independently and simultaneously — the Robot performs automated actions, while the User can intervene or manually manage the interface. To enable parallel work, run automated actions in the background (for example, in a separate thread).

#### 5.2. Switching Control Between the Robot and User

In this case, the Robot first controls the interface, and then the User can take control from the Robot, or vice versa. To switch control, implement a flag or switch that stops the automated work or enables it, allowing the scenario to be managed manually or automatically.

## 6. Examples of PiP Work Scenarios in Practice

<table data-header-hidden><thead><tr><th width="59.04998779296875"></th><th width="172.63330078125"></th><th width="222.0833740234375"></th><th width="223.51666259765625"></th></tr></thead><tbody><tr><td><strong>No.</strong></td><td><strong>Scenario</strong></td><td><strong>Operation Principle</strong></td><td><strong>Goal</strong></td></tr><tr><td>1.</td><td>Data Processing + Manual Control</td><td>The Robot collects data while the User observes and intervenes if necessary</td><td>Ensuring control and quick response</td></tr><tr><td>2.</td><td>Interface Testing</td><td>The Robot automates repetitive actions while the User observes and checks</td><td>Quick verification without stopping automation</td></tr><tr><td>3.</td><td>Running a Scenario with Simultaneous Manual Adjustment</td><td>The User inputs or adjusts data during the Robot's Work</td><td>Speeding up work with boundary precision</td></tr></tbody></table>

## 7. FAQ on PiP Mode

**Question:** Can two Robots be launched simultaneously in PiP mode?

**Answer:** No. Only one Robot can be launched in PiP mode. This Robot can later launch other background Robots.

**Question:** What should I do if the Robot cannot interact with the interface?

**Answer:** Check the active window, access rights, and the presence of selectors. Often, this is related to incorrect settings or blocks.

**Question:** Can I work in PiP with browsers?

**Answer:** The browser must be launched in only one instance in PiP mode; otherwise, the plugin will not be able to work with the Robot.