logsearch_properties.md

December 5, 2018 · View on GitHub

Log Search Configurations

NameDescriptionDefaultExamples
hadoop.security.credential.provider.pathPath to interrogate for protected credentials. (see: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html)EMPTY
  • localjceks://file/home/mypath/my.jceks
logsearch.admin.kerberos.cookie.domainDomain for Kerberos cookie.localhost
  • c6401.ambari.apache.org
  • localhost
logsearch.admin.kerberos.cookie.pathCookie path of the kerberos cookie/
  • /
logsearch.admin.kerberos.token.valid.secondsKerberos token validity in seconds.30
  • 30
logsearch.auth.external_auth.enabledEnable external authentication (currently Ambari acts as an external authentication server).false
  • true
  • false
logsearch.auth.external_auth.host_urlExternal authentication server URL (host and port).http://ip:port
  • https://c6401.ambari.apache.org:8080
logsearch.auth.external_auth.login_urlLogin URL for external authentication server ($USERNAME parameter is replaced with the Login username)./api/v1/users/$USERNAME/privileges?fields=*
  • /api/v1/users/$USERNAME/privileges?fields=*
logsearch.auth.file.enabledEnable file based authentication (in json file at logsearch configuration folder).true
  • true
  • false
logsearch.auth.jwt.audiancesComma separated list of acceptable audiences for the JWT token.EMPTY
  • audiance1,audiance2
logsearch.auth.jwt.cookie.nameThe name of the cookie that contains the JWT token.hadoop-jwt
  • hadoop-jwt
logsearch.auth.jwt.enabledEnable JWT based authentication (e.g.: for KNOX).false
  • true
  • false
logsearch.auth.jwt.provider_urlURL to the JWT authentication server.EMPTY
  • https://c6401.ambari.apache.org:8443/mypath
logsearch.auth.jwt.public_keyPEM formatted public key for JWT token without the header and the footer.EMPTY
  • MIGfMA0GCSqGSIb3DQEBA...
logsearch.auth.jwt.query.param.original_urlName of the original request URL which is used to redirect to Log Search Portal.originalUrl
  • myUrl
logsearch.auth.jwt.user.agentsComma separated web user agent list. (Used as prefixes)Mozilla,Opera,Chrome
  • Mozilla,Chrome
logsearch.auth.ldap.base.dnBase DN of LDAP database.EMPTY
  • dc=apache,dc=org
logsearch.auth.ldap.enabledEnable LDAP based authentication (currenty not supported).false
  • true
  • false
logsearch.auth.ldap.group.role.attributeAttribute for identifying LDAP groups (group name)cn
  • cn
logsearch.auth.ldap.group.role.mapMap of LDAP groups to Log Search rolesEMPTY
  • ROLE_CUSTOM1:ROLE_USER,ROLE_CUSTOM2:ROLE_ADMIN
logsearch.auth.ldap.group.search.baseGroup search base - defines where to find LDAP groups. Won't do any authority/role mapping if this field is empty.EMPTY
  • ou=people
logsearch.auth.ldap.group.search.filterGroup search filter which is used to get membership data for a specific userEMPTY
  • (memberUid={0})
logsearch.auth.ldap.manager.dnDN of the LDAP manger user (it is a must if LDAP groups are used).EMPTY
  • cn=admin,dc=apache,dc=org
logsearch.auth.ldap.manager.passwordPassword of the LDAP manager user.EMPTY
  • mypassword
logsearch.auth.ldap.manager.password.fileFile that contains password of the LDAP manager user.EMPTY
  • /my/path/passwordfile
logsearch.auth.ldap.password.attributePassword attribute for LDAP authenticationuserPassword
  • password
logsearch.auth.ldap.referral.methodSet the method to handle referrals for LDAPignore
  • follow
logsearch.auth.ldap.role.prefixRole prefix that is added for LDAP groups (as authorities)ROLE_
  • ROLE_
logsearch.auth.ldap.urlURL of LDAP database.EMPTY
  • ldap://localhost:389
logsearch.auth.ldap.user.dn.patternDN pattern that is used during login (dn should contain the username), can be used instead of user filterEMPTY
  • uid={0},ou=people
logsearch.auth.ldap.user.search.baseUser search base for user search filterEMPTY
  • ou=people
logsearch.auth.ldap.user.search.filterUsed for get a user based on on LDAP search (username is the input), if it is empty, user dn pattern is used.EMPTY
  • uid={0}
logsearch.auth.proxyserver.ipIP of trusted Knox Proxy server(s) that Log Search will trust onEMPTY
  • 192.168.0.1,192.168.0.2
logsearch.auth.proxyuser.groupsList of user-groups which trusted-proxy user ‘knox’ can proxy for*
  • admin,user
logsearch.auth.proxyuser.hostsList of hosts from which trusted-proxy user ‘knox’ can connect from*
  • host1,host2
logsearch.auth.proxyuser.usersList of users which the trusted-proxy user ‘knox’ can proxy forknox
  • knox,hdfs
logsearch.auth.redirect.forwardForward redirects for HTTP calls. (useful in case of proxies)false
  • true
logsearch.auth.simple.enabledEnable simple authentication. That means you won't require password to log in.false
  • true
  • false
logsearch.auth.trusted.proxyA boolean property to enable/disable trusted-proxy 'knox' authenticationfalse
  • true
logsearch.authr.file.enabledA boolean property to enable/disable file based authorizationfalse
  • true
logsearch.authr.role.fileSimple file that contains user/role mappings.roles.json
  • logsearch-roles.json
logsearch.cert.algorithmAlgorithm to generate certificates for SSL (if needed).sha256WithRSA
  • sha256WithRSA
logsearch.cert.folder.locationFolder where the generated certificates (SSL) will be located. Make sure the user of Log Search Server can access it./usr/lib/ambari-logsearch-portal/conf/keys
  • /etc/mypath/keys
logsearch.config.api.enabledEnable config API feature and shipperconfig API endpoints.true
  • false
logsearch.config.api.filter.solr.enabledUse solr as a log level filter storagefalse
  • true
logsearch.config.api.filter.zk.enabledUse zookeeper as a log level filter storagefalse
  • true
logsearch.config.zk_aclsZooKeeper ACLs for handling configs. (read & write)world:anyone:cdrwa
  • world:anyone:r,sasl:solr:cdrwa,sasl:logsearch:cdrwa
logsearch.config.zk_connect_stringZooKeeper connection string.EMPTY
  • localhost1:2181,localhost2:2181/znode
logsearch.config.zk_connection_retry_time_out_msThe maximum elapsed time for connecting to ZooKeeper in milliseconds. 0 means retrying forever.EMPTY
  • 1200000
logsearch.config.zk_connection_time_out_msZooKeeper connection timeout in millisecondsEMPTY
  • 30000
logsearch.config.zk_rootZooKeeper root node where the shippers are stored. (added to the connection string)EMPTY
  • /logsearch
logsearch.config.zk_session_time_out_msZooKeeper session timeout in millisecondsEMPTY
  • 60000
logsearch.hadoop.security.auth_to_localRules that will be applied on authentication names and map them into local usernames.DEFAULT
  • RULE:[1:\$1@\$0](.*@EXAMPLE.COM)s/@.*//
  • DEFAULT
logsearch.http.header.access-control-allow-credentialsAccess-Control-Allow-Credentials header for Log Search Server.true
  • true
  • false
logsearch.http.header.access-control-allow-headersAccess-Control-Allow-Headers header for Log Search Server.origin, content-type, accept, authorization
  • content-type, authorization
logsearch.http.header.access-control-allow-methodsAccess-Control-Allow-Methods header for Log Search Server.GET, POST, PUT, DELETE, OPTIONS, HEAD
  • GET, POST
logsearch.http.header.access-control-allow-originAccess-Control-Allow-Origin header for Log Search Server.*
  • *
  • http://c6401.ambari.apache.org
logsearch.http.portLog Search http port61888
  • 61888
  • 8888
logsearch.https.portLog Search https port61889
  • 61889
  • 8889
logsearch.jetty.access.log.enabledEnable jetty access logsfalse
  • true
logsearch.login.credentials.fileName of the credential file which contains the users for file authentication (see: logsearch.auth.file.enabled).user_pass.json
  • logsearch-admin.json
logsearch.protocolLog Search Protocol (http or https)http
  • http
  • https
logsearch.roles.allowedComma separated roles for external authentication.AMBARI.ADMINISTRATOR,CLUSTER.ADMINISTRATOR
  • AMBARI.ADMINISTRATOR
logsearch.session.timeoutLog Search http session timeout in minutes.30
  • 300
logsearch.solr.audit.logs.alias.nameAlias name for audit log collection (can be used for Log Search audit collection and ranger collection as well).audit_logs_alias
  • audit_logs_alias
logsearch.solr.audit.logs.collectionName of Log Search audit collection.audit_logs
  • audit_logs
logsearch.solr.audit.logs.config.nameSolr configuration name of the audit collection.audit_logs
  • audit_logs
logsearch.solr.audit.logs.config_set.folderLocation of Log Search audit collection configs for Solr./usr/lib/ambari-logsearch-portal/conf/solr_configsets
  • /usr/lib/ambari-logsearch-portal/conf/solr_configsets
logsearch.solr.audit.logs.numshardsNumber of Solr shards for audit collection (bootstrapping).1
  • 2
logsearch.solr.audit.logs.replication.factorSolr replication factor for audit collection (bootstrapping).1
  • 2
logsearch.solr.audit.logs.urlURL of Solr (non cloud mode) - currently unsupported.EMPTY
  • localhost1:8868
logsearch.solr.audit.logs.zk.aclsList of Zookeeper ACLs for Log Search audit collection (Log Search and Solr must be able to read/write collection details)EMPTY
  • world:anyone:r,sasl:solr:cdrwa,sasl:logsearch:cdrwa
logsearch.solr.audit.logs.zk_connect_stringZookeeper connection string for Solr (used for audit log collection).EMPTY
  • localhost1:2181,localhost2:2181/mysolr_znode
logsearch.solr.config_set.folderLocation of Solr collection configs./usr/lib/ambari-logsearch-portal/conf/solr_configsets
  • /usr/lib/ambari-logsearch-portal/conf/solr_configsets
logsearch.solr.implicit.routingUse implicit routing for Solr Collections.false
  • true
logsearch.solr.implicit.routingUse implicit routing for Solr Collections.false
  • true
logsearch.solr.jaas.filePath of the JAAS file for Kerberos based Solr Cloud authentication./usr/lib/ambari-logsearch-portal/logsearch_solr_jaas.conf
  • /my/path/jaas_file.conf
logsearch.solr.kerberos.enableEnable Kerberos Authentication for Solr Cloud.false
  • true
  • false
logsearch.solr.metadataName of Log Search metadata collection.logsearch_metadata
  • logsearch_metadata
logsearch.solr.metadata.config.nameSolr configuration name of the logsearch metadata collection.logsearch_metadata
  • logsearch_metadata
logsearch.solr.metadata.numshardsNumber of Solr shards for logsearch metadta collection (bootstrapping).2
  • 3
logsearch.solr.metadata.replication.factorSolr replication factor for event metadata collection (bootstrapping).2
  • 3
logsearch.solr.metadata.schema.fields.populate.interval.minsInterval in minutes for populating schema fiels for metadata collections.1
  • 10
logsearch.solr.ranger.audit.logs.collectionName of Ranger audit collections (can be used if ranger audits managed by the same Solr which is used for Log Search).EMPTY
  • ranger_audits
logsearch.solr.service.logsName of Log Search service log collection.hadoop_logs
  • hadoop_logs
logsearch.solr.service.logs.config.nameSolr configuration name of the service log collection.hadoop_logs
  • hadoop_logs
logsearch.solr.service.logs.numshardsNumber of Solr shards for service log collection (bootstrapping).1
  • 2
logsearch.solr.service.logs.replication.factorSolr replication factor for service log collection (bootstrapping).1
  • 2
logsearch.solr.urlURL of Solr (non cloud mode) - currently unsupported.EMPTY
  • localhost1:8868
logsearch.solr.zk.aclsList of Zookeeper ACLs for Log Search Collections (Log Search and Solr must be able to read/write collection details)EMPTY
  • world:anyone:r,sasl:solr:cdrwa,sasl:logsearch:cdrwa
logsearch.solr.zk_connect_stringZookeeper connection string for Solr.EMPTY
  • localhost1:2181,localhost2:2181/mysolr_znode
logsearch.spnego.kerberos.enabledEnable SPNEGO based authentication for Log Search Server.false
  • true
  • false
logsearch.spnego.kerberos.hostlocalhost
  • c6401.ambari.apache.org
  • localhost
logsearch.spnego.kerberos.keytabKeytab for SPNEGO authentication for Http requests.EMPTY
  • /etc/security/keytabs/mykeytab.keytab
logsearch.spnego.kerberos.principalPrincipal for SPNEGO authentication for Http requestsEMPTY
  • myuser@EXAMPLE.COM
logsearch.web.audit_logs.component.labelsMap of component component labels.ambari:Ambari,hdfs:Hdfs,RangerAudit:Ranger
  • ambari:Ambari,RangerAudit:ranger
logsearch.web.audit_logs.field.common.excludesList of fields that will be excluded from metadata schema responses for every audit components.tags,tags_str,seq_num
  • reqUser,resp,tag_str
logsearch.web.audit_logs.field.common.filterable.common.excludesList of fields that will be excluded from filter selection on the UI for every audit components.EMPTY
  • tag_str,resp,tag_str
logsearch.web.audit_logs.field.common.labelsMap of fields labels for audits (common).enforcer:Access Enforcer,access:Access Type,cliIP:Client Ip,cliType:Client Type,dst:DST,evtTime:Event Time,ip:IP,logtime:Log Time,sess:Session,ugi:UGI,reqUser:User,repo:Audit Source
  • reqUser:Req User,resp:Response
logsearch.web.audit_logs.field.common.visibleList of fields that will be displayed by default on the UI for every audit components.access,cliIP,evtTime,repo,resource,result,reqUser
  • reqUser,resp
logsearch.web.audit_logs.field.excludesList of fields that will be excluded from metadata schema responses for different audit components.EMPTY
  • ambari:reqUser,resp,hdfs:ws_user,ws_role
logsearch.web.audit_logs.field.filterable.excludesList of fields that will be excluded from filter selection on the UI for different audit components.EMPTY
  • ambari:tag_str,resp,tag_str;RangerAudit:path,ip
logsearch.web.audit_logs.field.filterable.excludesEnable label fallback. (replace _ with spaces and capitalize properly)true
  • false
logsearch.web.audit_logs.field.labelsMap of fields (key-value pairs) labels for different component types.EMPTY
  • ambari#reqUser:Ambari User,ws_response:Response;RangerAudit#reqUser:Req User
logsearch.web.audit_logs.field.visibleList of fields that will be displayed by default on the UI for different audit components.EMPTY
  • ambari:reqUser,resp;RangerAudit:reqUser,repo
logsearch.web.labels.service_logs.field.fallback.prefixesList of prefixes that should be removed during fallback of audit field labels.ws_,std_
  • ws_,std_,sdi_
logsearch.web.labels.service_logs.field.fallback.prefixesList of prefixes that should be removed during fallback of service field labels.ws_,sdi_,std_
  • ws_,std_,sdi_
logsearch.web.labels.service_logs.field.fallback.suffixesList of suffixes that should be removed during fallback of audit field labels._i,_l,_s,_b
  • _i,_l,_s,_b
logsearch.web.labels.service_logs.field.fallback.suffixesList of suffixes that should be removed during fallback of service field labels._i,_l,_s,_b
  • _i,_l,_s,_b
logsearch.web.service_logs.component.labelsMap of serivce component labels.EMPTY
  • ambari_agent:Ambari Agent,ambari_server:Ambari Servcer
logsearch.web.service_logs.field.excludesList of fields that will be excluded from metadata schema responses.id,tags,text,message,seq_num,case_id,bundle_id,rowtype,event_count
  • seq_num,tag
logsearch.web.service_logs.field.filterable.excludesList of fields that will be excluded from filter selection on the UI.EMPTY
  • path,method,logger_name
logsearch.web.service_logs.field.labelsMap of serivce field labels.log_message:Message,type:Component,logtime:Log Time,thread_name:Thread
  • log_message:Message,ip:IP Address
logsearch.web.service_logs.field.visibleList of fields that will be displayed by default on the UI.log_message,level,logtime,type
  • log_message,path,logtime
logsearch.web.service_logs.group.labelsMap of serivce group labelsEMPTY
  • ambari:Ambari,yarn:YARN