ConfigMgr CCMRecentlyUsedApps blank or mtrmgr.log with StartPrepDriver - OpenService Failed with Error Issue

Issue:  ConfigMgr Clients which should be reporting via hardware inventory "CCMRecentlyUsedApps" have nothing to report.  An analysis of the client indicates there is nothing in WMI root\cimv2\sms\ccm_recentlyusedapps TO report, and mtrmgr.log on the client contains lines like "StartPrepDriver - OpenService Failed with Error".  See --> KB3213242 <-- at Microsoft for more details.  UPDATE to script on 2018-01-31

Remediation: What worked for us was to re-register the 'prepdrv.inf', and then restart SMS Agent Host (aka ccmexec)

Before you do anything suggested below--confirm this will fix the issue you are seeing.  Login to a box or two with the issue, and from an elevated cmd prompt, run the "remediation" powershell script below.  Watch mtrmgr.log; and manually check that root/cimv2/sms select * from ccm_recentlyusedapps gets information.  Once you see info, do a hardware inventory, and confirm that box now reports information up to your database, as you expect it to.  If manually remediating works, then you can look to completing the steps below to automate the fix across your environment.

What we did to remediate the 'StartPrepDriver - OpenService Failed with Error' using a Configuration Item and Baseline was this:

It is assumed that you are using a "custom client agent setting" for enabling CCM_RecentlyUsedApps, since you normally do NOT want to target "every client".  Usually you don't want to target heavily used Application Servers, or Citrix Servers--since hundreds of userIDs can 'launch' applications, the results of CCMRUA on those types of machines often won't process (the MIF file will be bigger than 50mb) and isn't that useful anyway.  So go check, in your Custom Client Agent Settings, which collection you are targeting to enable 'ccm recently used apps'.  Once you know that, then continue.

1) first, determine with SQL query how big of an issue it might be; maybe it was only a couple of boxes; and you can address them manually. Depending upon the count returned, it'll be up to you if you want to pursue this as a Configuration Item, or manually address.

   Declare @CollID as nvarchar(8) = (Select collectionid from v_collection c where c.name = 'The Collection Name You figured out has CCMRUA as a HINV rule')
   select count(ws.resourceid) [Count]
   from v_gs_Workstations_Status ws
   where ws.LastHWScan is not null
   and ws.resourceid not in (select resourceid from v_gs_ccm_recently_used_apps)
   and ws.resourceid in (select resourceid from v_fullcollectionmembership_valid fcm where [email protected])

2) Create the Collection Query, to target.

   - Create a new Collection, using the "limit to collection" as the collection you use for targeting when CCM Recently Used Apps information should be reported via Hardware Inventory. (the one you figured out above)

   - The collection query rule should have two conditions... where:
      SMS_G_System_WORKSTATION_STATUS.LastHardwareScan is not null
      and
      SMS_R_SYSTEM.ResourceId not in (Select ResourceId from SMS_G_System_CCM_RECENTLY_USED_APPS)

3) If the count of the SQL query, and the count of the Collection query are the same (or close enough); then you can continue to creating the ConfigItem, and deploying the Baseline to remediate the issue.

4) Create the Configuration Item, where the Detection and Remediation Logic are at the end of this blog post.  Both are Powershell scripts. for "what means compliant", it will be a String, equals "Compliant". Make sure you check the box about Remediate if non-compliant.

5) After you create the Configuration Item, create a Baseline, add the CI to the baseline, and deploy the baseline, with remediation, to the collection you created above. (you may want to do a pilot, to a subset of 2 or 3 specific boxes, just to be sure it will all work as you expect it to work)

UPDATE 2018-02-02:  Discovered that machines were "re-breaking".  So I ended up targeting 'every box' with the Remediation script--it'll only remediate if problem found.  Note in my production remediation script, I comment out the ccmexec restart.  CCMEval takes care of that overnight, so it's OK.

6) That's it... then it's just a waiting game.  You are waiting for the Baseline to run, remediate the issue by doing the rundll routine, restart ccmexec.  Once that is done, in a minute or two mtrmgr.log will start to begin recording information into the root\cimv2\ccm_recentlyusedapps WMI location.  Once there is information there, then at the next hardware inventory, presuming this client is asked via hinv policy to report that information, it will have something to report.  So how long it takes completely depends upon YOUR schedules, that you defined.  How frequently the Baseline evaluates, and how frequently your clients do the scheduled HINV action.

PS: you might be tempted to "let me just add a line to the remediation script, to trigger a hinv right at the end".  I wouldn't bother.  It takes a few minutes for the client, after the ccmexec restart, to populate WMI.  It'll all shake out quickly enough.

######### DETECTION Powershell Script for the CI #####################
<#
.SYNOPSIS
   This Script is for Detection of a known condition for the inability of CM Client to initialize the StartPrepDriver
.DESCRIPTION
   This script finds the CM mgrmgr.log file, and reads it for a known good, or known error, condition.
   "what means good" =  mtrmgr.log contains lines like this
        PREP driver successfully initialized 
        or
        Termination event received for process
   "what means bad" = mtrmgr.log contains lines like this
        StartPrepDriver - OpenService Failed with error
.NOTES
   Steps are to
    1) read the regkey for CM Client to find the correct log file location
    2) Look for good or Bad entries in the file
    3) Exit with 'Compliant' or 'Non-Compliant' depending upon the results
.VERSION
    1.0 Sherry Kissinger  2017-03-30
1.1 Sherry Kissinger 2018-01-31 (added -Tail 10 for better accuracy)
#>
$ErrorActionPreference = 'SilentlyContinue'
#Get the LogDirectory for CM Client
$CMLogDir = (Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\CCM\Logging\@Global" -Name LogDirectory).LogDirectory
#Define the LogFile to Read
$CMLogFile = $CMLogDir + '\mtrmgr.log'
#Read the Log file for the Error Phrase
$GotErrors = (Get-Content -Path $CMLogFile -Tail 10 | where-object {$_ -like '*StartPrepDriver - OpenService Failed with error*'})
#Read the Log File for a known good condition phrases
$GotGoodEntry = (Get-Content -Path $CMLogFile -Tail 10 | where-object {($_ -like '*PREP driver successfully initialized*') -or ($_ -like '*Termination event received for process*')})
if ($GotGoodEntry) {
  write-host 'Compliant'
  }
else {
  if ($GotErrors) {
     write-host 'Non-Compliant'
  }
}

############## REMEDIATION Powershell Script for the CI  ##########################
<#
.SYNOPSIS
   This Script is for Detection and Remediation of a known condition for the inability of CM Client to initialize the StartPrepDriver
.DESCRIPTION
   This script finds the CM mgrmgr.log file, and reads it for a known good, or known error, condition.
   "what means good" =  mtrmgr.log contains lines like this
        PREP driver successfully initialized 
        or
        Termination event received for process
   "what means bad" = mtrmgr.log contains lines like this
        StartPrepDriver - OpenService Failed with error
.NOTES
   Steps are to
    1) read the regkey for CM Client to find the correct log file location
    2) Look for good or Bad entries in the file
    3) Exit with 'Compliant' or if 'Non-Compliant', to run the fix
       3a) the fix is two steps
            RUNDLL32.EXE SETUPAPI.DLL,InstallHinfSection DefaultInstall 128 C:\WINDOWS\CCM\prepdrv.inf
            Restart-Service ccmexec
.VERSION
    1.0 Sherry Kissinger  2017-03-30
1.1 Sherry Kissinger 2018-01-31 (added -Tail 10 for better accuracy)
#>
$ErrorActionPreference = 'SilentlyContinue'
#Get the LogDirectory for CM Client
$CMLogDir = (Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\CCM\Logging\@Global" -Name LogDirectory).LogDirectory
#Define the LogFile to Read
$CMLogFile = $CMLogDir + '\mtrmgr.log'
#Read the Log file for the Error Phrase
$GotErrors = (Get-Content -Path $CMLogFile -Tail 10 | where-object {$_ -like '*StartPrepDriver - OpenService Failed with error*'})
#Read the Log File for a known good condition phrases
$GotGoodEntry = (Get-Content -Path $CMLogFile -Tail 10 | where-object {($_ -like '*PREP driver successfully initialized*') -or ($_ -like '*Termination event received for process*')})
if ($GotGoodEntry) {
  write-host 'Compliant'
  }
else {
if ($GotErrors) {
 Try { Set-ExecutionPolicy -ExecutionPolicy 'Bypass' -Scope 'Process' -Force -ErrorAction 'Stop'}
 Catch {}
 $CMClientDIR = (Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\SMS\Client\Configuration\Client Properties" -Name 'Local SMS Path').'Local SMS Path'
 $ExePath = $env:windir + '\system32\RUNDLL32.EXE'
 $CLine = ' SETUPAPI.DLL,InstallHinfSection DefaultInstall 128 ' + $CMClientDIR + 'prepdrv.inf'
 #Parse the Parameters
 $Prms = $Cline.Split(" ")
 #Execute the fix with parameters
 & "$Exepath" $Prms
 #Restart ccmexec service
 #CCMExec should be restarted; If you'd rather wait for a natural client reboot, comment out this line.
 #Note that this CI will continue to attempt to remediate until a ccmexec restart or reboot
 restart-service ccmexec
  }
}

 

 

CM12

  • Created on .

ConfigMgr Current Branch Topic ID 611 swd State Messages flood

Issue:  Clients are flooding the inboxes\auth\statesys.box\incoming with state messages similar to the below.  There isn't much there to go on. The only promising thing to look for was that it's "Topic ID="611"" and Type="611".  Researching those topicid of 611 and we found nothing public, and opening a case with Microsoft provided some clues, but this was really a new issue, at least at this scale.

<?xml version="1.0" encoding="UTF-16"?>
<Report>
<ReportHeader>
<Identification>
  <Machine>
   <ClientInstalled>1</ClientInstalled>
   <ClientType>1</ClientType>
   <ClientID>GUID:0EE65490-9075-41B1-B64D-AAAAAAAAAAAA</ClientID>
   <ClientVersion>5.00.8412.1006</ClientVersion>
   <NetBIOSName>CLIENTNAMEHERE</NetBIOSName>
   <CodePage>437</CodePage>
   <SystemDefaultLCID>1033</SystemDefaultLCID>
   <Priority>5</Priority>
  </Machine>
</Identification>
<ReportDetails>
   <ReportContent>State Message Data</ReportContent>
   <ReportType>Full</ReportType>
   <Date>20170314180133.967000+000</Date>
   <Version>1.0</Version>
   <Format>1.0</Format>
</ReportDetails>
</ReportHeader>
<ReportBody>
<StateMessage MessageTime="20170314180133.857000+000" SerialNumber="18823">
  <Topic ID="611" Type="611" IDType="0" User="" UserSID=""/>
  <State ID="100" Criticality="0"/>
  <UserParameters Flags="0" Count="2">
   <Param>GUID:0EE65490-9075-41B1-B64D-AAAAAAAAAAAA</Param>
   <Param>0</Param>
  </UserParameters>
</StateMessage>
</ReportBody>
</Report>

Cause: In a ConfigMgr 1610 environment, and in 1602, 1606 versions of this were available as well, the cause of these messages is because that client is a member of a collection, where either by accident or design, that collection has the setting for "All devices are part of the same server group".  The collection contained 40% of all clients in the environment--and in our case checking the box was NOT by design--it was accidental.

What that does, is two things we observed (and others have documented 1 of them, see links below). 
1) as per one of the links below, machines in that collection may not ever patch as expected, again.  that's because it thinks it's part of a cluster, and if it's not... it's waiting for it's "turn" to patch. 
2) Every single device in that collection, once per minute, locally does two "schedule triggers", for two different things:
{00000000-0000-0000-0000-000000000111} -- which is for "Send Unsent State Message"
{00000000-0000-0000-0000-000000000116} -- which is for "State system policy bulk send low"

You'll see that over and over and over again locally on the client in SMSClientMethodProvider.log

and that apparently ends up as .swd files in the statesys box to be processed by the database, with TopicID 611, Type 611.  If it's enough devices, and it's hotfix Tuesday and lots of state messages anyway (*cough* for example *cough*)--the auth\statesys.box inbox may become backlogged, and never catch up.

Remediation:

I'm a SQL person, so using this sql query, identify the collections which have that checkbox checked--and confirm you really meant it (If you are here reading this blog post... you likely didn't mean it).  If not, uncheck the box for "All devices are part of the same server group" on the collections listed.

;with UseCluster as (select c.SiteID as [CollectionID] from CEP_CollectionExtendedProperties CEP
join collections_g c on CEP.Collectionid=c.Collectionid
where usecluster=1)
select c.*
from UseCluster
join v_collection c on c.collectionid=UseCluster.CollectionID

Monitoring for if this happens again:

https://mnscug.org/blogs/sherry-kissinger/503-example-of-custom-sql-job-to-log-to-application-event-viewer-for-configmgr

Links:
https://blogs.technet.microsoft.com/enterprisemobility/2016/05/16/update-1605-for-configuration-manager-technical-preview-available-now/
https://docs.microsoft.com/en-us/sccm/core/servers/deploy/install/release-notes
https://docs.microsoft.com/en-us/sccm/core/get-started/capabilities-in-technical-preview-1605#BKMK_ServerGroups
https://social.technet.microsoft.com/Forums/en-US/86783d86-0e38-4cb4-acf8-6110acc76c0e/configmgr-1602-error-0x87d006662016410010-while-installing-update?forum=configmanagersecurity

 

  • Created on .

Blog Series: 30 days of Intune Trial, Part 3

Still no software in the software report.  I scrounged up a Windows 10 Home Tablet, and am trying to get that registered with Intune.  I can add the account, but on the client tablet when I go to "work access" for the account, and "Enroll in to device management" it tells me that

"System Policies prevent you from connecting to a work or school account.  Contact your support person for more information.  "

So I'm the support person I'm sure--so hmm... time to Bing and Google and see what is preventing me from actually enrolling a Windows 10 Home tablet; the BYOD scenario...

And Google-fu tells me that apparently one needs to be a local Administrator.  I thought I was; but I see that once I logged into my LiveID; the Windows 10 box decided I was a standard user, not an Admin locally on the box.  Had to promote that user to an Administrator, reboot, and then I could complete the enrollment.

Now the Company Portal actually shows me things.  Still nothing in the Software report.  But the box was just enrolled.  I'll have to check on it later. 

  • Created on .

Blog series: 30 days of Intune Trial, Part 2

Day 2:

A nice Microsoft person contacted me directly, just to be sure I had everything I needed for this trial.  I told him yes; I'm just testing and getting familiar with it.  If I hit some horrid snag I'll just go ask in the CM forums.

The policy made yesterday for "configuration" was Android Policies; and it was targeted to a group that contained a test user, not a device grouping.

Downloaded the Company Portal on an Android phone (Samsung S4), signed in with the user account created yesterday; the one which deserves the Configuraton Policy.  That ConfigPolicy I made custom--with only 2 very easy-to-see settings. After signing in, and going through next/next/finish on the phone, confirmed the 2 changes were enforced.

Even with currently only 1 device--much of my work life is creating reports.  Over the years, I just got good at 'knowing' where in the database information was stored, and how best to extract it when parameters change.  So in Manage.Microsoft.com, there's a reporting section.  Now that I have 1 thing to look at; time to start poking about.

Created a report for "all Devices, All publishers, All Categories".  This was created at least an hour after the device got policies, but there are zero results.  Is there something additional I need to configure?  Or does this only work for certain devices, and Android isn't one of these devices?  Or is that report only for Apps as deployed by Intune--which I haven't deployed any yet.

I do see the 1 device in the "Mobile Device Inventory Reports".  So maybe it's just time--I need to wait for the device to do an inventory and send it up. 

Since I can't see any software yet in reporting, just recording the portals that I've so far had to get to:
https://portal.office.com -- Office 365
https://portal.azure.com -- to create users and usergroups for Intune standalone to leverage
https://manage.microsoft.com -- the main Intune (at least how I think of Intune) portal.

Added another mobile device--this time a Galaxy Tab 4; I'm sure it'll get the policies just fine.  I'm more curious about what I see in reports.

Just some user experience stuff, when signing into the Company Portal, the "Activate Device Administrator" lists a lot of scary-sounding permissions; but that's only because I'm so used to Android devices being unmanaged/wide open.  But I've also become jaded to the 'permissions needed' on Android since the latest updates to the Android OS.  It's kind of like click "I agree" to those legal license notifications--you just don't read them anymore.

Well, I've waited about 3 hours since having those 2 devices get policies--and still nothing in "Detected Software Reports".  I'll let them hang out overnight--maybe that's a "every xx hours" thing or something, and I'm just too impatient. 

  • Created on .

Blog series: 30 days of Intune Trial, Part 1

When Intune was beta, I tried a trial at the time--which was well over a year ago.  Since things change--including me--I decided to try another 30 day trial.

Back when I was looking at it over a year ago; my priority at the time was to see if it could be capable of 'fitting in' with the needs of managing a full 'pro' or 'business' version of Windows 8.  The mobile management pieces of it weren't that interesting to me; it wasn't part of the landscape for me at work, and quite honestly I didn't have personal devices to test with.  I still don't have a full range and scope of personal test devices; but tally-ho anyway!

Here's my new premise (subject to change of course--this is testing).
- Use Intune Standalone only, not hybrid to the home lab
- Get familiar with the console
- Reporting
- Policies
- Management: users or devices or combination

I know that Intune works--it clearly works fine.  It's my journey to understand how it works with my on-premise ConfigMgr background and knowledge.  I'm sure it'll be a learning curve for me.

Day 1:
Signed up for Intune 30 day trial, created the first username and password.
It asked me to sign in to https://portal.office.com, but it wasn't letting me login at first-- had to sign out of my real liveID, and then it told me the This email address is being protected from spambots. You need JavaScript enabled to view it. account didn't exist.
Then I just made a new IE tab; and pasted in https://portal.office.com -- and without asking for a password, logged me right in.  Seems a little odd; but I suppose that was due to the liveID.

There's an online walkthrough; it told me to create a new group, but any group name I tried said "failed to add group"
Call to b__9f was not permitted with the token|$$|ContextClass=AdHoc|$$|0=b__9f|$$|

Again, not very helpful.  But after much clicking around, found that portal.office.com wasn't the right place to add a group--apparently I need to do that over on portal.azure.com; which is then available to be targeted by 'things' in manage.microsoft.com.  Seems like a lot of consoles to go to... but if I think of it in my "how does similar stuff work when on premise"--one has a console for AD users and computers, and the CM console, and other consoles--so having multiple consoles to do similar setup for Intune shouldn't surprise me.  Might also be because I *am* standalone testing, and not Azure AD / Hybrid testing.  That would be a different experience and work flow.

Created a test user; and was able to create a group, and explicitly add that test user to that group.
Created a very very basic Configuration policy for Android, and targeted that group.

End of day one... Summary:
Created 30 day trial, created a standard user, created a group with that user, created and deployed a policy to that group.

  • Created on .

Copyright © 2018 - The Minnesota System Center User Group