Remove Site to Crawl

Audience: Low-code Engineers

Skill Prerequisites: Actions, Tokens

Removes a site from the web crawler, so it isn't crawled anymore. The site and all its queued pages, with their crawl history, are deleted.

You find the site by the Site Id you gave it in Add Site to Crawl. For the whole crawl flow, see How crawling works.

note

This action is part of the Web Crawler add-on (PlantAnApp.WebCrawler). The add-on is installed separately and needs the WEBCRWL feature in your license. If it isn't licensed, the action fails with a "not licensed" error. If you don't see the Web Crawler actions, the add-on isn't installed.

Typical Use Cases

• Stop crawling a site that's no longer needed
• Let users remove a site they added from a form or a listing
• Clean up test sites after trying the crawler

Don't use it to

• Crawl a site again from the start. Run Add Site to Crawl again with the same Site Id instead. It clears the queue and starts over.
• Delete the pages you saved while crawling. The crawler doesn't know about your own tables. Delete them with Run SQL Query.
• Pause a crawl. There's no pause. Disable the automation job that runs Crawl Next Batch instead.

Related Actions

Action Name	Description
Add Site to Crawl	Adds a site and its crawl rules to the queue.
Crawl Next Batch	Fetches the next queued pages and runs actions for each one.
Run SQL Query	Deletes the content you saved for the site.

Input Parameter Reference

Parameter	Description	Supports Tokens	Default	Required
Site Id	The Site Id the site was added with, for example ExampleDocs. It's not the internal number from Output Site Id Token Name.	Yes	empty string	Yes

Output Parameters Reference

This action doesn't create any tokens.

Considerations

• Errors. The action fails when Site Id is empty (There is no token defined for the UserAssignedSiteId), or when no site has this Site Id (There is no site with the specified '<Site Id>' site id.). Add a condition, or wrap it in Execute Actions, if the site may already be gone.
• Sites without a Site Id can't be removed. Sites added without a Site Id can't be found by this action. Always set a Site Id in Add Site to Crawl.
• It can't be undone. The crawl history is lost. If you add the site again, every page is crawled again from the start.
• Any site. The Site Id is global. The action removes the site whichever portal or module added it. If users can type the Site Id, check that they're allowed to remove that site.
• Remove it between batches. Avoid removing a site while Crawl Next Batch is running, for example by removing it from the same job, before or after that action.

Examples

tip

To understand how to use the below examples, please see Running Examples.

1. Remove a site

This action removes the site added with the Site Id ExampleDocs, and all its queued pages.

{
    "Title": "Remove Site to Crawl",
    "ActionType": "WebCrawler.RemoveSiteToCrawl",
    "Description": "Stop crawling the example documentation",
    "Parameters": {
        "UserAssignedSiteId": "ExampleDocs"
    }
}

2. Remove the site of a listing row

In a listing of sites, this action removes the site of the selected row, only when the row has a Site Id. The site's saved pages are then deleted from your own CrawledPages table.

[
    {
        "Title": "Remove Site to Crawl",
        "ActionType": "WebCrawler.RemoveSiteToCrawl",
        "Description": "Stop crawling the selected site",
        "Condition": "[SiteId] != \"\"",
        "Parameters": {
            "UserAssignedSiteId": "[SiteId]"
        }
    },
    {
        "Title": "Run SQL Query",
        "ActionType": "RunSql",
        "Description": "Delete the saved pages of the site",
        "Condition": "[SiteId] != \"\"",
        "Parameters": {
            "SqlQuery": "DELETE FROM CrawledPages WHERE SiteId = @SiteId",
            "BindTokens": [
                {
                    "name": "SiteId",
                    "value": "[SiteId]"
                }
            ]
        }
    }
]

Revised 09/28/2026