How to check complete list of kubernetes objects?

 Following command successfully display all kubernetes objects

kubectl api-resources

Example

[root@hsk-controller ~]# kubectl api-resources

NAME                              SHORTNAMES       KIND

bindings                                           Binding

componentstatuses                 cs               ComponentStatus

configmaps                        cm               ConfigMap

endpoints                         ep               Endpoints

events                            ev               Event

limitranges                       limits           LimitRange

namespaces                        ns               Namespace

nodes                             no               Node

persistentvolumeclaims            pvc              PersistentVolumeClaim

persistentvolumes                 pv               PersistentVolume

pods                              po               Pod

podtemplates                                       PodTemplate

replicationcontrollers            rc               ReplicationController

resourcequotas                    quota            ResourceQuota

secrets                                            Secret

serviceaccounts                   sa               ServiceAccount

services                          svc              Service

initializerconfigurations                          InitializerConfiguration

mutatingwebhookconfigurations                      MutatingWebhookConfiguration

validatingwebhookconfigurations                    ValidatingWebhookConfiguration

customresourcedefinitions         crd,crds         CustomResourceDefinition

apiservices                                        APIService

controllerrevisions                                ControllerRevision

daemonsets                        ds               DaemonSet

deployments                       deploy           Deployment

replicasets                       rs               ReplicaSet

statefulsets                      sts              StatefulSet

tokenreviews                                       TokenReview

localsubjectaccessreviews                          LocalSubjectAccessReview

selfsubjectaccessreviews                           SelfSubjectAccessReview

selfsubjectrulesreviews                            SelfSubjectRulesReview

subjectaccessreviews                               SubjectAccessReview

horizontalpodautoscalers          hpa              HorizontalPodAutoscaler

cronjobs                          cj               CronJob

jobs                                               Job

brpolices                         br,bp            BrPolicy

clusters                          rcc              Cluster

filesystems                       rcfs             Filesystem

objectstores                      rco              ObjectStore

pools                             rcp              Pool

certificatesigningrequests        csr              CertificateSigningRequest

leases                                             Lease

events                            ev               Event

daemonsets                        ds               DaemonSet

deployments                       deploy           Deployment

ingresses                         ing              Ingress

networkpolicies                   netpol           NetworkPolicy

podsecuritypolicies               psp              PodSecurityPolicy

replicasets                       rs               ReplicaSet

nodes                                              NodeMetrics

pods                                               PodMetrics

networkpolicies                   netpol           NetworkPolicy

poddisruptionbudgets              pdb              PodDisruptionBudget

podsecuritypolicies               psp              PodSecurityPolicy

clusterrolebindings                                ClusterRoleBinding

clusterroles                                       ClusterRole

rolebindings                                       RoleBinding

roles                                              Role

volumes                           rv               Volume

priorityclasses                   pc               PriorityClass

storageclasses                    sc               StorageClass

volumeattachments                                  VolumeAttachment

Note: kubernate version is v1.12*

How To Install Helm3 on Windows10

 Introduction

Helm is a tool that streamlines installing and managing Kubernetes applications.Think of it like apt/yum/homebrew for Kubernetes.

- Helm renders your templates and communicates with the Kubernetes API

- Helm runs on your laptop, CI/CD, or wherever you want it to run.

-Helm3 bring lot more capabilities and feature

Deploying applications to Kubernetes is a complicated process. Many tools simplify this process, and one of them is Helm.

Prerequisites

A system running with window 10

Access to a command line/terminal

A Kubernetes cluster installed and configured

To download Helm on Windows:

1. Follow the link below to download the latest Helm version.
https://github.com/helm/helm/releases

2. Locate the Windows amd64 download link from the Installation platform list and click on it to download.

3. Next, extract the windows-amd64 zip to the preferred location.

4.Example->C:\Users\hss\Downloads\helm-v3.5.2-windows-amd64\windows-amd64

5. Set enviromnet variable and point to helm.exe

6. Your done....Goto command line and start executng helm commands

C:\Users\>helm version

version.BuildInfo{Version:"v3.4.2", GitCommit:"167aac70832d3a384f65f9745335e9fb40169dc2", GitTreeState:"dirty", GoVersion:"go1.15.7"}

Happy helming!

java.lang.StringIndexOutOfBoundsException: String index out of range: -12 in CDH Upgrade

scheduledTime=2018-11-21T10:30:00.000Z}.
2018-11-21 16:00:00,054 INFO 336151100@scm-web-19:com.cloudera.server.web.common.ExceptionReport: (63 skipped) Exception report generated accessing https://10.43.230.133:7183/cmf/services/landingPageStatusContent
java.lang.StringIndexOutOfBoundsException: String index out of range: -12
at java.lang.String.substring(String.java:1967)
at com.cloudera.cmf.model.DbHostHeartbeat.checkJavaComponent(DbHostHeartbeat.java:177)
at com.cloudera.cmf.model.DbHostHeartbeat.getActiveComponentInfo(DbHostHeartbeat.java:207)
at com.cloudera.cmf.service.HostUtils.hasComponent(HostUtils.java:291)
at com.cloudera.server.web.common.Util.isCDHCluster(Util.java:144)
at com.cloudera.server.web.cmf.menu.ClusterActionsMenuHelper.addClusterCommandMenu(ClusterActionsMenuHelper.java:81)
at com.cloudera.server.web.cmf.menu.ClusterActionsMenuHelper.<init>(ClusterActionsMenuHelper.java:36)
at com.cloudera.server.web.cmf.AggregateStatusController.getClusterModel(AggregateStatusController.java:1039)
at com.cloudera.server.web.cmf.AggregateStatusController.servicesTable(AggregateStatusController.java:575)
at sun.reflect.GeneratedMethodAccessor1361.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:585)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)
at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:131)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at com.jamonapi.http.JAMonServletFilter.doFilter(JAMonServletFilter.java:48)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter.doFilter(JavaMelodyFacade.java:109)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:125)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.handler.StatisticsHandler.handle(StatisticsHandler.java:53)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2018-11-21 16:00:00,732 INFO com.cloudera.cmf.scheduler-1_Worker-1:com.cloudera.cmf.scheduler.CommandDispatcherJob: Skipping scheduled command 'GlobalPoolsRefresh' since it is a noop.
2018-11-21 16:00:30,569 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=19ms, min=1ms, max=156ms.
2018-11-21 16:01:30,614 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=19ms, min=1ms, max=156ms.
2018-11-21 16:02:30,646 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=19ms, min=1ms, max=156ms.
2018-11-21 16:03:30,684 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:04:28,478 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2018-11-21 16:04:28,480 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2016-11-21T10:34:28.478Z to reap.
2018-11-21 16:04:28,481 INFO StaleEntityEviction:com.cloudera.server.cmf.node.NodeScannerService: Reaped 0 requests.
2018-11-21 16:04:28,482 INFO StaleEntityEviction:com.cloudera.server.cmf.node.NodeConfiguratorService: Reaped 0 requests.
2018-11-21 16:04:29,353 INFO 1495743394@agentServer-2:com.cloudera.server.common.MonitoringThreadPool: agentServer: execution stats: average=20ms, min=3ms, max=181ms.
2018-11-21 16:04:29,353 INFO 1495743394@agentServer-2:com.cloudera.server.common.MonitoringThreadPool: agentServer: waiting in queue stats: average=0ms, min=0ms, max=15ms.
2018-11-21 16:04:30,716 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:05:30,745 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:06:30,772 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:07:30,811 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:08:30,854 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:09:30,916 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:10:30,950 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:11:30,986 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=18ms, min=1ms, max=156ms.
2018-11-21 16:12:31,037 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:13:31,085 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:14:28,501 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2018-11-21 16:14:28,503 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2016-11-21T10:44:28.502Z to reap.
2018-11-21 16:14:28,505 INFO StaleEntityEviction:com.cloudera.server.cmf.node.NodeScannerService: Reaped 0 requests.
2018-11-21 16:14:28,505 INFO StaleEntityEviction:com.cloudera.server.cmf.node.NodeConfiguratorService: Reaped 0 requests.
2018-11-21 16:14:29,754 INFO 1495743394@agentServer-2:com.cloudera.server.common.MonitoringThreadPool: agentServer: execution stats: average=18ms, min=10ms, max=73ms.
2018-11-21 16:14:29,754 INFO 1495743394@agentServer-2:com.cloudera.server.common.MonitoringThreadPool: agentServer: waiting in queue stats: average=0ms, min=0ms, max=1ms.
2018-11-21 16:14:31,127 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:14:43,120 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: (119 skipped) ScmActive completed successfully.
2018-11-21 16:14:50,326 WARN pool-180-thread-1:com.cloudera.server.web.cmf.StatusProvider: (144 skipped) Failed to submit task for getting status from SERVICE_MONITORING
com.cloudera.cmon.MgmtServiceNotRunningException: SERVICE_MONITORING is not running
at com.cloudera.cmon.MgmtServiceLocator.getNozzleIPC(MgmtServiceLocator.java:145)
at com.cloudera.server.web.cmf.StatusProvider$SubjectStatusCustomFuture.<init>(StatusProvider.java:618)
at com.cloudera.server.web.cmf.StatusProvider.getStatus(StatusProvider.java:1043)
at com.cloudera.server.web.cmf.AggregateStatusController$HealthInfo.<init>(AggregateStatusController.java:887)
at com.cloudera.server.web.cmf.AggregateStatusController$HealthInfo.<init>(AggregateStatusController.java:855)
at com.cloudera.server.web.cmf.AggregateStatusController$1.load(AggregateStatusController.java:176)
at com.cloudera.server.web.cmf.AggregateStatusController$1$1.call(AggregateStatusController.java:188)
at com.cloudera.server.web.cmf.AggregateStatusController$1$1.call(AggregateStatusController.java:185)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-11-21 16:14:55,759 WARN EventStorePublisherWithRetry-0:com.cloudera.cmf.event.publish.EventStorePublisherWithRetry: Failed to publish event: SimpleEvent{attributes={CATEGORY=[AUDIT_EVENT], EVENTCODE=[EV_LOGIN_SUCCESS], SERVICE=[ClouderaManager], USER=[admin], SERVICE_TYPE=[ManagerServer], MESSAGE_CODES=[LOGIN_SUCCESS], SEVERITY=[INFORMATIONAL]}, content=User admin logged in successfully., timestamp=1542795294547} - 1 of 343 failure(s) in last 1801s
2018-11-21 16:15:01,328 WARN pool-180-thread-1:com.cloudera.server.web.cmf.StatusProvider: (123 skipped) Failed to submit task for getting status from HOST_MONITORING
com.cloudera.cmon.MgmtServiceNotRunningException: HOST_MONITORING is not running
at com.cloudera.cmon.MgmtServiceLocator.getNozzleIPC(MgmtServiceLocator.java:145)
at com.cloudera.server.web.cmf.StatusProvider$SubjectStatusCustomFuture.<init>(StatusProvider.java:618)
at com.cloudera.server.web.cmf.StatusProvider.getStatus(StatusProvider.java:1034)
at com.cloudera.server.web.cmf.AggregateStatusController$HealthInfo.<init>(AggregateStatusController.java:887)
at com.cloudera.server.web.cmf.AggregateStatusController$HealthInfo.<init>(AggregateStatusController.java:855)
at com.cloudera.server.web.cmf.AggregateStatusController$1.load(AggregateStatusController.java:176)
at com.cloudera.server.web.cmf.AggregateStatusController$1$1.call(AggregateStatusController.java:188)
at com.cloudera.server.web.cmf.AggregateStatusController$1$1.call(AggregateStatusController.java:185)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-11-21 16:15:01,337 INFO 1270484339@scm-web-32:com.cloudera.server.web.common.ExceptionReport: (82 skipped) Exception report generated accessing https://10.43.230.133:7183/cmf/services/landingPageStatusContent
java.lang.StringIndexOutOfBoundsException: String index out of range: -12
at java.lang.String.substring(String.java:1967)
at com.cloudera.cmf.model.DbHostHeartbeat.checkJavaComponent(DbHostHeartbeat.java:177)
at com.cloudera.cmf.model.DbHostHeartbeat.getActiveComponentInfo(DbHostHeartbeat.java:207)
at com.cloudera.cmf.service.HostUtils.hasComponent(HostUtils.java:291)
at com.cloudera.server.web.common.Util.isCDHCluster(Util.java:144)
at com.cloudera.server.web.cmf.menu.ClusterActionsMenuHelper.addClusterCommandMenu(ClusterActionsMenuHelper.java:81)
at com.cloudera.server.web.cmf.menu.ClusterActionsMenuHelper.<init>(ClusterActionsMenuHelper.java:36)
at com.cloudera.server.web.cmf.AggregateStatusController.getClusterModel(AggregateStatusController.java:1039)
at com.cloudera.server.web.cmf.AggregateStatusController.servicesTable(AggregateStatusController.java:575)
at sun.reflect.GeneratedMethodAccessor1361.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:585)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:595)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78)
at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:131)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at com.jamonapi.http.JAMonServletFilter.doFilter(JAMonServletFilter.java:48)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter.doFilter(JavaMelodyFacade.java:109)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:125)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.handler.StatisticsHandler.handle(StatisticsHandler.java:53)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2018-11-21 16:15:28,270 INFO CMMetricsForwarder-0:com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder: (29 skipped) Failed to send metrics.
java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy133.writeMetrics(Unknown Source)
at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.sendWithAvro(ClouderaManagerMetricsForwarder.java:325)
at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.sendMetrics(ClouderaManagerMetricsForwarder.java:312)
at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.run(ClouderaManagerMetricsForwarder.java:146)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.avro.AvroRemoteException: java.net.ConnectException: Connection refused (Connection refused)
at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:88)
... 11 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1220)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:984)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
at org.apache.avro.ipc.HttpTransceiver.writeBuffers(HttpTransceiver.java:71)
at org.apache.avro.ipc.Transceiver.transceive(Transceiver.java:58)
at org.apache.avro.ipc.Transceiver.transceive(Transceiver.java:72)
at org.apache.avro.ipc.Requestor.request(Requestor.java:147)
at org.apache.avro.ipc.Requestor.request(Requestor.java:101)
at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:72)
... 11 more
2018-11-21 16:15:31,171 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:16:31,215 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:17:31,253 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:18:31,276 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=48ms.
2018-11-21 16:19:31,322 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=38ms.
2018-11-21 16:20:31,364 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=38ms.
2018-11-21 16:21:31,396 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=38ms.
2018-11-21 16:22:31,423 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=17ms, min=9ms, max=39ms.
2018-11-21 16:23:31,473 INFO avro-servlet-hb-processor-3:com.cloudera.server.common.AgentAvroServlet: (35 skipped) AgentAvroServlet: heartbeat processing stats: average=16ms, min=8ms, max=39ms.
2018-11-21 16:24:28,524 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2018-11-21 16:24:28,526 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2016-11-21T10:54:28.524Z to reap.
2018-11-21 16:24:28,527 INFO StaleEntityEviction:com.cloudera.server.cmf.node.NodeScannerService: Reaped 0 requests.
2018-11-21 16:24:28,527 INFO StaleEntityEviction:com.cloudera.server.cmf.node.NodeConfiguratorService: Reaped 0 requests.
2018-11-21 16:24:30,179 INFO 1495743394@agentServer-2:com.cloudera.server.common.MonitoringThreadPool: agentServer: execution stats: average=17ms, min=8ms, max=73ms.
2018-11-21 16:24:30,179 INFO 1495743394@agentServer-2:com.cloudera.server.common.MonitoringThreadPool

Hadoop 3 works with HDFS Erasure Coding

Hadoop 3 add HDFS Erasure Coding

Well Hadoop 3 already available with improvements including support for HDFS Erasure coding, a preview of v2 of the YARN Timeline Service, and improvements to YARN/HDFS federation.

Now a day Hadoop is a framework is used to process large data sets across clusters of computers using simple programming models. 

The addition of HDFS erasure coding should make data more durable and to reduce the amount of storage needed for HDFS. 

The default three times replication scheme in HDFS has a 200 per cent  overhead in storage space and other resources such as network bandwidth.

For many datasets with relatively low I/O activities, additional block replicas are rarely accessed during normal operations, but still consume the same amount of resources as the first replica. If Erasure Coding is used in place of replication, the storage overhead is no more than 50 per cent. HDFS Erasure Coding uses RAID , in which Erasure Coding is implemented by stripping. This logically stores the data in the form of a block, and stores the block on the different disk. For each block, the parity is calculated and stored. This is the encoding, and any error can be recovered by back calculating using the parity.

The new release also includes a preview of the YARN Timeline Service v.2, which offers better scalability, reliability, and usability of the Timeline Service. The service is responsible for persisting application specific information, and for persisting generic information about completed applications.

Understand HDFS Erasure Coding 

Erasure Coding helps Capacity Utilization & Performance for Data Storage Systems

HDFS by default replicates each block three times. Replication provides a simple and robust form of redundancy to shield against most failure scenarios. It also eases scheduling compute tasks on locally stored data blocks by providing multiple replicas of each block to choose from.

However, replication is expensive: the default 3x replication scheme incurs a 200% overhead in storage space and other resources (e.g., network bandwidth when writing the data). For datasets with relatively low I/O activity, the additional block replicas are rarely accessed during normal operations, but still consume the same amount of storage space.

Therefore, a natural improvement is to use erasure coding (EC) in place of replication, which uses far less storage space while still providing the same level of fault tolerance. Under typical configurations, EC reduces the storage cost by ~50% compared with 3x replication

Advantages of Erasure Coding in Hadoop

• Saving substantial space – Initially, blocks are triplicated when they are sealed (no longer modified), a background task encode it and delete it replicas.
• Flexible policy – User and admin able to flag the file hot and cold. Hot files are replicated even after it sealed.
• Fast Recovery – HDFS block errors are discovered and recovered both actively (in the background) and passively (on the read path).
• Low overhead – Because of parity bit overhead is up to 50%.
• Transparency/compatibility – HDFS user should be able to use all basic and advanced features on erasure coded data, including snapshot, encryption, appending, caching and so forth.

YARN Improvements

YARN is a framework for job scheduling and cluster resource management, and high availability for the HDFS filing system.

YARN federation is used to scale single YARN clusters to tens of thousands of nodes, by federating multiple YARN sub-clusters.

Support for YARN resource types has also been added, making it possible to schedule additional resources such as disks and GPUs for better integration with machine learning and container workloads.


Other improvements include the ability to federate YARN and HDFS subclusters transparently; and opportunistic container execution to improve resource utilization and increase task throughput for short-lived containers. Support for cloud storage systems such as Amazon S3  and Azure Data Lake has also been improved.

Apache Software Foundation announce Apache Hadoop 3.0.0 GA!

Wow Apache Hadoop reaches 3.0

!!Hadoop was born around 2007, and by 2017 its Part of Life!!

The Apache Software Foundation has announced version three of the open source software framework for distributed computing. 

It incorporates over 6,000 changes since its started over a year ago of the open source software framework for distributed computing. 
Apache Hadoop 3.0 is the first major release since Hadoop 2 was released in 2013.

“Hadoop 3 is a major milestone for the project, and our biggest release ever,” said Andrew Wang, Apache Hadoop 3 release manager. “It represents the combined efforts of hundreds of contributors over the five years since Hadoop 2. I’m looking forward to how our users will benefit from new features in the release that improve the efficiency, scalability, and reliability of the platform.”

Apache Hadoop has become known for its ability to run and manage data applications on large hardware clusters in the Big Data ecosystem. The latest release features HDFS erasure coding, a preview of YARN Timeline Service version 2, YARN resource types, and improved capabilities and performance enhancements around cloud storage systems. It includes Hadoop Common for supporting other Hadoop modules, the Hadoop Distributed File System, Hadoop YARN and Hadoop MapReduce.

“This latest release unlocks several years of development from the Apache community,” said Chris Douglas, vice president of Apache Hadoop. “The platform continues to evolve with hardware trends and to accommodate new workloads beyond batch analytics, particularly real-time queries and long-running services. At the same time, our Open Source contributors have adapted Apache Hadoop to a wide range of deployment environments, including the Cloud.”

Apache Hadoop is widely deployed in enterprises and companies like Adobe, AWS, Apple, Cloudera, eBay, Facebook, Google, Hortonworks, IBM, Intel, LinkedIn, Microsoft, Netflix and Teradata. 

In addition, it has inspired other Hadoop related projects such as: Apache Cassandra, HBase, Hive, Spark and ZooKeeper.

All you want To Know About Apache Hadoop 3.0.0

!!!Apache Hadoop 3.0: Major changes!!

HDFS erasure coding

halves the storage cost of HDFS while also improving data durability;

Minimum required Java version increased to Java 8

All Hadoop JARs are now compiled targeting a runtime version of Java 8, which means that those of you who are still using Java 7 or below should upgrade to Java 8.

Early preview of YARN Timeline Service major revision

Hadoop 3.0 also brings an early preview (alpha 2) of a major revision of YARN Timeline Service: v.2, which addresses two major challenges:

• improving scalability and reliability of Timeline Service
• enhancing usability by introducing flows and aggregation

Shell script rewrite

The Hadoop shell scripts have been rewritten to fix many long-standing bugs and include some new features. However, keep in mind that some changes could break existing installations.You’ll find the incompatible changes in the release notes, with related discussion on HADOOP-9902.

MapReduce task-level native optimization

MapReduce has added support for a native implementation of the map output collector. For shuffle-intensive jobs, this can lead to a performance improvement of 30% or more.

Shaded client jars

The hadoop-client Maven artifact available in 2.x releases pulls Hadoop’s transitive dependencies onto a Hadoop application’s classpath. This can be problematic if the versions of these transitive dependencies conflict with the versions used by the application.


Over the past decade, Apache Hadoop has become ubiquitous within the greater Big Data ecosystem by enabling firms to run and manage data applications on large hardware clusters in a distributed computing environment.

Default ports of multiple services have been changed.

Previously, the default ports of multiple Hadoop services were in the Linux ephemeral port range (32768-61000). This meant that at startup, services would sometimes fail to bind to the port due to a conflict with another application.

Apache Hadoop 3.0.0 highlights:

• YARN Timeline Service v.2 —improves the scalability, reliability, and usability of the Timeline Service;
• YARN resource types —enables scheduling of additional resources, such as disks and GPUs, for better integration with machine learning and container workloads;
• Federation of YARN and HDFS subclusters transparently scales Hadoop to tens of thousands of machines;
• Opportunistic container execution improves resource utilization and increases task throughput for short-lived containers. In addition to its traditional, central scheduler, YARN also supports distributed scheduling of opportunistic containers; and 
• Improved capabilities and performance improvements for cloud storage systems such as Amazon S3 (S3Guard), Microsoft Azure Data Lake, and Aliyun Object Storage System.

Hadoop 3.0.0 release Details:

After four alpha releases and one beta release, 3.0.0 is generally available. 3.0.0 consists of 302 bug fixes, improvements, and other enhancements since 3.0.0-beta1. All together, 6242 issues were fixed as part of the 3.0.0 release series since 2.7.0.

Here is series of alpha and beta releases leading up to an eventual Hadoop 3.0.0 GA.

3.0.0-alpha1	2016-09-03
3.0.0-alpha2	2017-01-25
3.0.0-alpha3	2017-05-26
3.0.0-alpha4	2017-07-07
3.0.0-beta1	2017-10-03
3.0.0 GA	2017-12-13

"Hadoop Is Here To Stay" —Forrester

Basic Postgres Command to list databases and list Schema

How to List Databases and Tables in PostgreSQL Using psql

First login to PostgreSQL using psql command and run following command from PostgreSQL command prompt.

 psql -U postgres

PostgreSQL command line cheatsheet:

Some interesting flags (to see all, use -h):

• -E: will describe the underlaying queries of the \ commands (cool for learning!)
• -l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)

Most \d commands support additional param of __schema__.name__ and accept wildcards like *.*

• \q: Quit/Exit
• \c __database__: Connect to a database
• \d __table__: Show table definition including triggers
• \dt *.*: List tables from all schemas (if *.* is omitted will only show SEARCH_PATH ones)
• \l: List databases
• \dn: List schemas
• \df: List functions
• \dv: List views
• \df+ __function__ : Show function SQL code.
• \x: Pretty-format query results instead of the not-so-useful ASCII tables

Listing Databases

A single Postgres server process can manage multiple databases at the same time. Each database is stored as a separate set of files in its own directory within the server’s data directory. To view all of the defined databases on the server you can use the \list meta-command or its shortcut \l.

Listing Tables

Once you’ve connected to a database, you will want to inspect which tables have been created there. This can be done with the \dt meta-command. However, if there are no tables you will get no output.

postgres=# connect sampledb

sampledb=# dt

More Example Coming Soon..

Fencing Method for ZK based HA in Hadoop

While enabling  ZK quorum based HA via ZKFC service you can observed that default fencing method (i.e sshfence ) alone is not sufficient .Details about scenarios and fencing methods Discussed below:

dfs.ha.fencing.methods - a list of scripts or Java classes which will be used to fence the Active NameNode during a failover

The fencing methods used during a failover are configured as a carriage-return-separated list, which will be attempted in order until one indicates that fencing has succeeded. There are two methods which ship with Hadoop: shell and sshfence. For information on implementing your own custom fencing method, see the org.apache.hadoop.ha.NodeFencer class.

sshfence - SSH to the Active NameNode and kill the process

The sshfence option SSHes to the target node and uses fuser to kill the process listening on the service’s TCP port. In order for this fencing option to work, it must be able to SSH to the target node without providing a passphrase. Thus, one must also configure the dfs.ha.fencing.ssh.private-key-files option, which is a comma-separated list of SSH private key files. For example:

    <property>

      <name>dfs.ha.fencing.methods</name>

      <value>sshfence</value>

    </property>

    <property>

      <name>dfs.ha.fencing.ssh.private-key-files</name>

      <value>/home/exampleuser/.ssh/id_rsa</value>

    </property>

shell - run an arbitrary shell command to fence the Active NameNode

The shell fencing method runs an arbitrary shell command. It may be configured like so:

    <property>

      <name>dfs.ha.fencing.methods</name>

      <value>shell(/path/to/my/script.sh arg1 arg2 ...)</value>

    </property>

The string between ‘(’ and ‘)’ is passed directly to a bash shell and may not include any closing parentheses.

If the shell command returns an exit code of 0, the fencing is determined to be successful. If it returns any other exit code, the fencing was not successful and the next fencing method in the list will be attempted.

Note: This fencing method does not implement any timeout. If timeouts are necessary, they should be implemented in the shell script itself (eg by forking a subshell to kill its parent in some number of seconds)

Scenario #1

If namenode process killed or Active Name Node rebooted

  Then passive node using ZKFC process able to make passive namenode  state ‘active’ using (sshfence)

  ssh work fine which allow to fence active namenode

Scenario #2

If namenode shutdown/node cable unplugged/ major hardware/power failure

    Then ssh will not happen and HA will never work .

    SSH fail so fencing of active Namenode failed

  

Best Approach

To avoid Scenario #2 one can use both fencing method which will be tried by ZKFC in the order of: if the first one fails, the second one will be tried.

 if ssh fencing failed then shell(/bin/true) will always true and fence active namenode.

Example:

<value>sshfence

       shell(/bin/true)

</value>

Property TAG:

<name>dfs.ha.automatic-failover.enabled</name>

<value>true</value>

<name>dfs.ha.fencing.methods</name>

<value>sshfence

       shell(/bin/true)

</value>

Tested this approach and it worked fine specially when Primary NN host down due to major hardware/power failure as mentioned.