Benchmarking
Insights ranks your numbers against every other FireMon Insights customer, with no matching by industry, size or region. A count on its own tells you what you have. The benchmark tells you how that count compares with everyone else running the same product.
The comparison needs every customer's measurements in one place. Only FireMon's cloud has them, whatever version of SIP you run.
None of these numbers is a target set by FireMon. Each one shows where you sit among other customers.
Where the comparison appears
Insights draws the comparison as a column, a line or a marker, depending on the page.
Industry Comparison, FireMon Customer Average (FCA) and Similar Organizations all name the same set of customers.
| Where | What you see | Range behind the comparison |
|---|---|---|
| All Metrics | An Industry Comparison column — your percentile rank for that metric, as a bar with the number on it | Fixed at the last 30 days |
| Metric charts | The FireMon Customer Average for that metric, against your own value | Fixed at the last 30 days |
| Best Practices | The FCA for each category, and how far above or below it your score sits | The range you selected on the page |
| The KPI carousel | On a metric expressed as a proportion, a third bar labeled Similar Organizations — the median across customers — beside your current and starting values | Fixed at the last 30 days |
The customer average line is drawn on the full-size metric chart: on metric detail pages, and on the Total Rules and Tickets Created cards on Revision Analytics. The smaller chart on a KPI tile has no line.
A KPI tile shows the All Metrics percentile rather than calculating its own. KPI covers the tile's Percentile and Trend views.
Everything is compared at the all-devices level
Every one of these comparisons, drawn or quoted, is built from each customer's device group 1, the all-devices group. Your own side of the comparison isn't. It follows whichever device group you selected.
So the two sides only match when you're looking at all devices. Narrow to a single device group and you're comparing part of your devices with all of everyone else's: your DMZ firewalls against other customers' entire installations. Insights still calculates the number and still shows it. Device groups covers where groups come from and what else they scope.
Counts and proportions
A raw count would rank a large installation badly just for being large. So where a metric has a natural total, both sides of the comparison are converted to a share of that total before anything is ranked. Unused rules are compared as a share of total rules, control failures by severity as a share of total rules, unreferenced network objects as a share of total network objects, and so on.
A customer with 40,000 rules and 8,000 unused is compared at 20%. A customer with 4,000 rules and 1,200 unused is compared at 30%. The first ranks lower despite having nearly seven times as many unused rules, and on a metric where low is the ideal, lower is better.
Not every metric has a total to divide by. Total Devices, Total Security Rules and Total Network Objects have nothing to be a share of. Neither do the two scores, SCI and complexity, which already sit on a scale of their own, or counts like Number of Revisions and User Logins. Those are ranked on the raw number. Where the raw number rises with the size of your installation, so does the ranking.
The Revision Analytics baseline is always a proportion. It is each customer's change in total rule count across the range, as a percentage of what they started the range with.
Metrics covers when a count is expressed against a total, and how that carries into the change figures.
Reading a percentile
The percentile is the share of customers with a value lower than yours. Insights counts how many sit below you and how many sit above, then reports the ones below as a share of both counts added together, rounded to a whole number. Customers whose value is exactly the same as yours count toward neither, including your own record.
No metric in Insights is better high. Every metric definition either has "low" as its ideal, or has no ideal direction. That gives you two ways to read a percentile:
- Where low is the ideal — unused rules, shadowed rules, control failures by severity, duplicate and unreferenced objects, complexity — a low percentile is good. It means most customers carry more of that thing than you do.
- Where there is no ideal direction — Total Devices, Total Security Rules, Total Network Objects — the percentile only describes you. A 90th percentile on Total Devices means you manage more devices than most customers do.
Best Practices scores work the same way. A category score runs 0 to 10, where 0 means nothing is failing, so a score above the FCA marker is worse than average, not better. See Scores.
When a comparison is missing
| What you see | Why |
|---|---|
| A dash in the Industry Comparison column | Your own most recent batch has no value for that metric. On a metric compared as a proportion, a missing or zero total also counts as no value. A dash also appears when no other customer has a value that differs from yours, which leaves nothing to rank against |
| No customer average line on a metric chart | The average has no place on the chart as drawn: it falls outside the range your bars cover, or belongs on a percentage axis your series doesn't have. Insights drops the line rather than rescaling the chart |
| No FCA marker on a Best Practices category | Your account has no FireMon Best Practices Assessment, or none of its controls carry that category's tags, or no customer results were found for those controls over the range you selected |
A comparison missing on one metric and present on the next is normal. Each metric's set of customers is assembled separately.