Microsoft wants to add a new Office 365 feature to allow customers to test Microsoft Defender email protection without actually having to configure the environment and devices for your organization.
Microsoft Defender for Office 365 (previously known as Office 365 Advanced Threat Protection) provides Office 365 enterprise accounts with email protection from several threats including credential phishing and business email compromise, as well as automated remediation of detected attacks.
This new mode is “the most effective way to evaluate the filtering efficacy of Defender for Office 365 using real-world email in your production environment,” Microsoft explains.
Evaluate in production
The soon-to-be-released evaluation experience is designed to help admins focus on evaluating Microsoft Defender for Office 365 capabilities and on eliminating the complexities of environment and device configuration.
This means that, after enabling Defender for Office 365 evaluation mode in your environment, emails will get scanned and evaluated without having to make any changes in the Microsoft 365 admin center.
“When the evaluation mode is set up, you will have a report updated daily with up to 90 days of data quantifying the messages that would have been blocked if the policies were implemented (for example, delete, send to junk, quarantine),” Microsoft adds.
“They are aggregated per detection technology (for example, impersonation) and can be filtered by time range. Additionally, message reports can be created on-demand to create custom pivots or to deep dive messages using Threat Explorer.”
Defender for Office 365 email policies (e.g., Safe Attachments, Safe Links, and anti-phishing impersonation policies) created after toggling on evaluation mode will log verdicts, such as malware.
However, they will not act on analyzed emails in any way, allowing you to understand the security solution’s efficiency when dealing with real-world messages in production environments.
“The feature can be configured by all customers that have Microsoft Exchange Online mailboxes, and provides a report that quantifies the messages that would have been blocked by Defender for Office 365 had the policies taken action,” Microsoft says.
“After the evaluation report has been generated, see how many advanced threat links, advanced threat attachments, and potential impersonations were identified in the emails and collaboration workspaces in your organization.”
This evaluation mode requires a Billing admin or a Global admin role and it only applies to email protection features of Microsoft Defender for Office 365 — doesn’t apply to SharePoint, Office Clients, or Teams.
To access it you’ll need a Microsoft Defender for Office 365 Plan 1, Microsoft Defender for Office 365 Plan 2, Microsoft 365 E5 / Microsoft 365 E5 Security, or Office 365 E5 license.
To get started with the evaluation, you can use the set-up card in the Office 365 Security & Compliance center accessible via Threat management > Dashboard, Threat management > Policy, or Reports > Dashboard.
If you don’t have a compatible license, you can quickly start a free 30-day evaluation and test it in the Office 365 Security & Compliance Center.
The new Office 365 feature will soon be available in public preview, with Microsoft planning to make it generally available to all customers in standard multi-tenants with compatible licenses.
Microsoft has also recently launched Office 365 priority protection for accounts of high-profile employees such as executive-level managers who are regularly targeted in attacks.