How to troubleshoot incidents of endpoint vRO data collection embedded in HRV 7

Hello

We have problems with the collection of data for the endpoint vRO in vra 7. Data collection fails. We believe this is related to the worker DEM, but we do not know how to solve this. So far, the only mistake we have is when running on endpoint vRO data collection:

In the console of vRA > infrastructure > monitoring, logs:

Endpoint data collection has no endpoint internal Instance [Workflow Instance Id = 31009] vRO vRealize Orchestrator has returned an error: internal server error.

In C:\Program Files (x 86) execution of the \VMware\vCAC\Distributed Manager\DEM - Core2\Logs\DEM-Core2_Errors

[UTC:2016 - 05-20 21:24:05 local: 2016-05-20 14:24:05] "[Error]: Thread-Id ="11"- context =" "token =" "<? XML version = "1.0" encoding = "utf-16"? >

< Boolean > false < / Boolean >

Endpoint data collection has no endpoint internal Instance [Workflow Instance Id = 31009] vRO

System.Net.WebException: vRealize Orchestrator has returned an error: internal server error.

at DynamicOps.VcoModel.Common.VcoClient.CheckResponseStatus (IRestResponse answer)

at DynamicOps.VcoModel.Common.VcoClient.DecodeJsonResponse (IRestResponse answer)

to DynamicOps.VcoModel.Common.VcoVersionUtils.GetPluginsInformation (customer VcoClient)

to DynamicOps.VcoModel.Activities.StoreVcoPluginsInformation.Execute (CodeActivityContext context)

to System.Activities.CodeActivity.InternalExecute (ActivityInstance, executor, BookmarkManager bookmarkManager ActivityExecutor instance)

to System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody (ActivityExecutor executor, BookmarkManager bookmarkManager, location resultLocation)

In addition, when you run the data collection on an inventory of POSSIBLE computing resources, the following error occurs:

[UTC:2016 - 05-20 22:54:53 local: 2016-05-20 15:54:53] "[Error]: Thread-Id ="12"- context =" "token =" "<? XML version = "1.0" encoding = "utf-16"? >

< Boolean > false < / Boolean >

Workflow "vSphereVCNSInventory" failed with the following exception:

DynamicOps.VcoModel.Common.VcoException: Point endpoint could not be found. There is no point vRealize Orchestrator endpoints configured with the __VMware.VCenterOrchestrator.Plugin.NSX.Build property.

to DynamicOps.VcoModel.Common.EndpointSelector.ctor (context, ManagementModelEntities, String selectionFilter)

at DynamicOps.VCNSModel.Activities.VcoNsxHelper.VcoNsxConnectionInfo.GetVcoEndpoint (ManagementModelEntities managementDataContext, Guid virtualMachineId)

at DynamicOps.VCNSModel.Activities.VcoNsxHelper.VcoNsxConnectionInfo.GetConnection (ManagementModelEntities managementDataContext, point endpoint VCNSEndpoint, Guid virtualMachineId)

at DynamicOps.VCNSModel.Workflows.vSphereVCNSInventory_CompiledExpressionRoot.vSphereVCNSInventory_CompiledExpressionRoot_TypedDataContext3_ForReadOnly.__Expr21Get)

at DynamicOps.VCNSModel.Workflows.vSphereVCNSInventory_CompiledExpressionRoot.InvokeExpression (Int32 expressionId, IList 1 locations, ActivityContext activityContext)

to Microsoft.CSharp.Activities.CSharpValue'1.Execute (CodeActivityContext context)

to System.Activities.Runtime.ActivityExecutor.ExecuteInResolutionContext [T] (ActivityInstance parentInstance, activity 1 expressionActivity)

to System.Activities.InArgument'1.TryPopulateValue (LocationEnvironment targetEnvironment ActivityInstance activityInstance ActivityExecutor executor)

to System.Activities.ActivityInstance.InternalTryPopulateArgumentValueOrScheduleExpression (RuntimeArgument argument, Int32 nextArgumentIndex, ActivityExecutor executor, IDictionary 2 argumentValueOverrides, place resultLocation, Boolean isDynamicUpdate)

to System.Activities.ActivityInstance.ResolveArguments (ActivityExecutor executor, IDictionary 2 argumentValueOverrides, place resultLocation, Int32 startIndex)

to System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody (ActivityExecutor executor, BookmarkManager bookmarkManager, location resultLocation)

We use a distributed deployment of vRA 7 and have 2 devices with the built-in vro. Both devices show the service as running vco-server, and I can connect to each instance of vco using the orchestrator client. Also the asax component can load workflows vro. I am able to navigate to the inventory.

Also, we noticed this problem after one of our servers of iaas is dead on us. We had to rebuild a starting from scratch and shiv inside. It was the Manager node, then this is the EDO, DEM and Manager of services that run on it. Not sure if this is related, it's just that we noticed after. It could occurred earlier also...

Any suggestions on where to look to refine this would be greatly appreciated.

Thank you

Darrenoid

Thank you for your response.

I finally understood that this morning. I searched in the /var/log/vco/app-server/server.log on the vra devices and found errors like this:

2016-05-21 08:31:33.345 - 0700 [http-nio-127.0.0.1-8280-exec-10] ERROR {} [[restServlet]] Servlet.service () for servlet [restServlet] in the context of path [/vco] has thrown the exception [processing request failed, the nested exception is java.lang.NullPointerException] origin

java.lang.NullPointerException

at com.vmware.o11n.web.util.CommonUtil.getServerConfiguration(CommonUtil.java:44)

at com.vmware.o11n.web.plugins.PluginController.listAllPlugins(PluginController.java:116)

at sun.reflect.GeneratedMethodAccessor347.invoke (unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)

at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)

at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)

at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:806)

at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:729)

at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)

at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)

at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)

at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)

at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)

at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)

at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)

to org.springframework.security.web.FilterChainProxy$ VirtualFilterChain.doFilter (FilterChainProxy.java:316)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)