If you are working with Microsoft dynamics 365 commerce you first have to initialize commerce scale unit.  You can perform this initialization only in tier 2 environments and with either the Project Owner or Environment manager role.

While initializing   retail commerce scale unit (RCSU) you might face below issue on LCS where the error says

Headquarters is not responding to health check calls. Please verify that the headquarters environment is up and running and not in maintenance mode

Troubleshooting:

In deployment logs we found the exact error as below


Real-time Service client library call failed. CorrelationId: ‘7cd03fdc-3424-4da8-9cd5-75ce9c61fd63’. ApiName: ‘IsAlive’. MethodName: ‘IsAlive’. Language: ‘(null)’. Company: ”. ParameterCount: ‘0’. Ex: ‘Microsoft.Dynamics.Retail.Cdx.RealtimeServiceClientLibrary.Exceptions.RtsCommunicationException: Real-time Service call for method ‘IsAlive’ successfully authenticated with AAD (Azure Active Directory), but was blocked by Operations. This can happen when the AAD client ID (set in CommerceRuntime.config) being used isn’t mapped to a user in Operations, that mapped user has insufficient permissions in Operations to perform Real-time service operations, or the configured AOS URL is incorrect (set via serviceHostUrl in CommerceRuntime.config). —> System.ServiceModel.FaultException: Forbidden

What this error suggests that RTS (Real time service) is trying to connect to HQ (F&O) through azure active directory app configured using RetailServiceAccount user present in F&O.

Since RetailServiceAccount was not present RTS was failing.

You can check RTS status by pinging this URL 

https:// <scale unit id from lcs>-rs-staging.azurewebsites.net/healthcheck?testname=ping

If the status is red for retail service, you can check if User and App is configured in F&O.

So , the cause of this issue is missing RetailServiceAccount from userinfo table and cause for that can anything listed below

  1. Maintenance mode for your environment is enabled.
  2. DB movement between environments.
  3. Sometimes while deploying environments, it fails multiple times and then deploys successfully.

You can check if RetailServiceAccount removed from userInfo table by running following query

 select ID, NAME, ENABLE, NETWORKALIAS, NETWORKDOMAIN, ACCOUNTTYPE, COMPANY, DEFAULTPARTITION, LANGUAGE, HELPLANGUAGE, SID, OBJECTID, PREFERREDTIMEZONE from userinfo where ID = ‘RetailServiceAccount’

If query returns no result, then you can execute the following query in the AXDB to restore the RetailSystemAccount and mark it as system account.

SET NOCOUNT ON

— CREATE A TEMPORARY STORED PROCEDURE TO CREATE ROLES.

IF OBJECT_ID ( ‘TEMPDB..#RETAILDRIASSIGNROLE’, ‘P’ ) IS NOT NULL  

    DROP PROCEDURE #RETAILDRIASSIGNROLE; 

GO

CREATE PROC #RETAILDRIASSIGNROLE @USERID VARCHAR(100), @ROLENAME VARCHAR(100)

AS

       DECLARE @SECURITYROLERECID BIGINT;

       SET @SECURITYROLERECID = (SELECT TOP 1 RECID

              FROM SECURITYROLE

              WHERE SECURITYROLE.AOTNAME = @ROLENAME)

       IF @SECURITYROLERECID IS NULL

       BEGIN

              PRINT ‘ROLE ‘  + @ROLENAME  + ‘ DOES NOT EXIST IN THE ENVIRONMENT. ROLE WILL BE SKIPPED.’

       END

       ELSE

       BEGIN

              IF EXISTS(SELECT * FROM SECURITYUSERROLE WHERE SECURITYROLE =  @SECURITYROLERECID AND [USER_] = @USERID) OR (@SECURITYROLERECID IS NULL)

              BEGIN

                      PRINT ‘ROLE ‘  + @ROLENAME  + ‘ WITH RECID: ‘ + CAST(@SECURITYROLERECID AS VARCHAR) + ‘ ALREADY EXISTS FOR THE USER. ROLE WAS NOT ASSIGNED TO USER.’

              END

              ELSE

              BEGIN

                      PRINT ‘ADDING ROLE ‘  + @ROLENAME  + ‘ WITH RECID: ‘ + CAST(@SECURITYROLERECID AS VARCHAR) + ‘ TO USER ‘ + @USERID

                      INSERT INTO SECURITYUSERROLE(SECURITYROLE, [USER_], ASSIGNMENTMODE, ASSIGNMENTSTATUS) VALUES(@SECURITYROLERECID, @USERID, 1, 1)    

              END

       END

GO

DECLARE @RETAILSERVICEACCOUNT VARCHAR(100);

SET @RETAILSERVICEACCOUNT = ‘RetailServiceAccount’;

BEGIN TRAN

–CHECK IF THE USER REORD ALREADY EXISTS.

IF NOT EXISTS( SELECT * FROM USERINFO WHERE ID = @RETAILSERVICEACCOUNT)

BEGIN 

        PRINT ‘CREATING SYSTEM ACCOUNT: ‘ + @RETAILSERVICEACCOUNT

       — LCS PROD

       INSERT INTO   USERINFO (ID, NAME, NETWORKALIAS, NETWORKDOMAIN, ACCOUNTTYPE, COMPANY, DEFAULTPARTITION , ENABLE, LANGUAGE, HELPLANGUAGE, SID, OBJECTID, PREFERREDTIMEZONE)

       VALUES (@RETAILSERVICEACCOUNT, @RETAILSERVICEACCOUNT, ‘RetailServerSystemAccount@dynamics.com’, ‘https://sts.windows.net/&#8217;, 2,       ‘dat’, 1,     1,     ‘EN-US’, ‘EN-US’, ‘S-1-19-2668615710-2480941646-1684813103-2026890972-2302207704-2864208061-3273002221-3047239075-1510753007-4079435192’, ‘FA6B0156-76D9-4BC2-B618-68CB7A3BF599’, 58)

       IF EXISTS(SELECT 1 FROM SYS.COLUMNS WHERE NAME = N’ISMICROSOFTACCOUNT’ AND OBJECT_ID = OBJECT_ID(N’DBO.USERINFO’))

       BEGIN

              PRINT ‘SETTING ISMICROSOFTACCOUNT TO 1 FOR ‘ + @RETAILSERVICEACCOUNT

              EXEC(‘UPDATE USERINFO SET ISMICROSOFTACCOUNT = 1 WHERE ID = ”’ + @RETAILSERVICEACCOUNT + ””)

              PRINT @RETAILSERVICEACCOUNT + ‘ IS NOW A SYSTEM ACCOUNT’

       END

       ELSE

       BEGIN

              PRINT ‘ISMICROSOFTACCOUNT IS NOT AVAILABLE IN THIS ENVIRONMENT. PLEASE HAVE THE CUSTOMER UPDATE TO THE LASTEST PLATFORM VERSION AVAILABLE.’

       END

       PRINT ‘USER CREATED’

END

ELSE

BEGIN

       PRINT ‘AN RECORD ALREADY EXISTS IN TABLE USERINFO WITH ID: ‘+ @RETAILSERVICEACCOUNT

       PRINT ‘NO USER HAS BEEN CREATED.’

END

— CREATE ROLES.

PRINT ‘ASSIGNING ROLES TO ‘ + @RETAILSERVICEACCOUNT

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘RETAILSERVICE’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘LEDGERACCOUNTANT’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘CUSTINVOICEACCOUNTSRECEIVABLECLERK’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘CUSTINVOICEACCOUNTSRECEIVABLEMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘BATCHJOBMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘COLLECTIONLETTERCOLLECTIONSMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘INVENTCOSTACCOUNTANT’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘INVENTCOSTCOSTCLERK’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘INVENTMATERIALSMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘INVENTQUALITYCONTROLMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘INVENTRECEIVINGCLERK’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘INVENTSHIPPINGCLERK’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘JMGADVTIMEWORKER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘TRADEPURCHASINGMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘RETAILCATALOGMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘RETAILMERCHANDISINGMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘RETAILOPERATIONSMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘RETAILWAREHOUSEMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘RETAILWAREHOUSECLERK’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘TRADESALESCLERK’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘TRADESALESMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘TRADESALESREPRESENTATIVE’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘SMASERVICEDELIVERYMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘SUBLEDGERJOURNALTRANSFERMAINTAINROLE’ — THIS ROLE IS NOT AVAILABLE IN OLDER ENVIRONMENTS.

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘FBTAXACCOUNTANT_BR’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘VENDVENDORACCOUNTMANAGER’

EXEC #RETAILDRIASSIGNROLE @RETAILSERVICEACCOUNT, ‘VENDPURCHASINGAGENT’

PRINT ‘ROLES CREATED’

COMMIT TRAN

GO

IF OBJECT_ID ( ‘TEMPDB..#RETAILDRIASSIGNROLE’, ‘P’ ) IS NOT NULL

BEGIN

    DROP PROCEDURE #RETAILDRIASSIGNROLE;

END

GO