Crawling Solutions

Crawling is the foundation of Cartographer. It discovers every component in a Dataverse solution and catalogs it, building the metadata that summaries, user guides, and diagrams are generated from.

What Crawling Does

When you crawl a solution, Cartographer queries the Dataverse metadata APIs to discover every component that belongs to that solution. It reads the raw metadata for each component — table definitions, column schemas, form XML, view FetchXML, plugin step registrations, cloud flow definitions, and more — and stores a normalized record in the Component table.

Crawling is solution-scoped, meaning you choose exactly which solutions to crawl. This is not a full-environment blast — you control which solutions are documented and which are ignored.

Incremental Crawling

After the first crawl, every re-crawl is incremental. Cartographer compares each component's current source against what it captured last time, and only reprocesses the ones that actually changed. A component whose source hasn't changed since the last crawl is skipped, not rewritten.

Fast re-crawls

Re-crawl a solution that hasn't changed and the crawl log reports zero created, zero updated, and everything skipped — finished in a couple of seconds instead of rebuilding the whole picture from scratch.

Adding a Solution to Crawl

  1. 1

    Open Solution Manager

    In the Cartographer app, navigate to Solution Manager in the left sidebar.

  2. 2

    Click Add Solution

    Click the Add Solution button in the command bar. A panel will appear with a dropdown of all solutions in your environment.

  3. 3

    Select a solution

    Choose the solution you want to crawl from the dropdown. You can search by solution name. Only managed and unmanaged solutions are shown — default solutions (like the Common Data Service Default Solution) are excluded.

  4. 4

    Configure crawl settings

    Optionally set how often it should re-crawl — manual only, hourly, every six hours, daily or weekly. Daily is the default.

  5. 5

    Save and Crawl

    Click Save to add the solution. Then click Crawl Now to start the initial crawl. The progress indicator will show how many components are being discovered.

Solution Manager with Add Solution dialog

Component Types Discovered

Cartographer discovers components across your solution's data model, user interface, automation, web resources, and security configuration. Each type is processed differently to extract the most useful metadata.

Data Model

  • Tables
  • Columns
  • Relationships
  • Global Option Sets

User Interface

  • Forms
  • Views
  • Charts
  • Dashboards
  • Model-Driven Apps
  • Site Maps

Automation

  • Plugin Assemblies
  • Custom APIs
  • Plugin Steps
  • Cloud Flows (Automated)
  • Cloud Flows (Scheduled)
  • Cloud Flows (Instant)
  • Classic Workflows
  • Business Rules
  • Business Process Flows

Web Resources

  • JavaScript
  • HTML
  • CSS
  • Images (SVG, PNG, ICO)

Security & Configuration

  • Security Roles
  • Connection References
  • Environment Variables
Plugin assemblies and custom APIs are newly discovered. Previously, crawling a solution built around custom APIs reported no automation for them at all.

Crawl Frequency

You can configure how often Cartographer re-crawls each solution to keep documentation up to date.

FrequencyDescription
Manual OnlyCrawl only when you click "Crawl Now"
HourlyRe-crawls every hour
Every 6 HoursRe-crawls four times a day
DailyRe-crawls once every 24 hours. This is the default.
WeeklyRe-crawls once every 7 days
Scheduled crawls are picked up by a background flow that polls every 30 minutes and checks each solution's configured frequency against its last completed crawl — so 30 minutes is how often the check runs, not how often your solution gets crawled. After installing Cartographer, an administrator needs to bind the connection reference ("Cartographer Dataverse Connection") and turn on the two Cartographer flows before scheduled crawls will run at all.

Re-crawling and Removing Solutions

Re-crawling

Click Crawl Now on any solution to trigger an immediate re-crawl. Incremental crawling means unchanged components are skipped. Use Full Re-crawl if you want to discard all cached data and start fresh.

Removing a solution

To stop crawling a solution, select it in Solution Manager and click Remove. The confirmation dialog states how many component records will be deleted and requires typing DELETE to confirm. This removes the crawl configuration and all associated component records from the Cartographer database. It does not affect the actual solution in your environment.

Removing a solution from Cartographer permanently deletes all discovered components, summaries, and diagrams for that solution. This action cannot be undone.

Crawl Log

Every crawl operation is recorded in the Crawl Log. You can view it from the Solution Manager to see:

  • Start and end timestamps
  • Counts of components created, updated, skipped, and removed
  • Crawl duration
  • Any errors or warnings encountered
  • Whether the crawl was manual or scheduled
A component that has disappeared from the solution since the last crawl is marked Removed rather than deleted outright, so its history stays in the log.

Next Steps

Once your solution is crawled, every component already has a plain-English summary, generated automatically from what was discovered.

SummariesAutomation Map