Sunbird inQuiry
Facing issue in Installing flink jobs #109
Replies: 3 comments 6 replies
-
|
@krgauraw Can you please help us with this? Or Please add people who can help us with this? |
Beta Was this translation helpful? Give feedback.
-
|
@mohitga @gandham-santhosh we are trying to deploy Sunbird inquiry independently on our server. Can you please help us to get it done? |
Beta Was this translation helpful? Give feedback.
-
|
Async-questionset flink job is meant for migration. Can you try disabling it here? https://github.com/project-sunbird/sunbird-ed-installer/blob/d5d47e5c9b56111f629ad6ccaf431c414b60c7f9/helmcharts/inquirybb/charts/flink/values.yaml#L237 Check if the values under this kafka object is being passed |
Beta Was this translation helpful? Give feedback.
-
|
Yes @gandham-santhosh please help in this @mohitga |
Beta Was this translation helpful? Give feedback.
-
|
@AbhilashKD what is the error from failing pod? Can you describe and check k8s events? |
Beta Was this translation helpful? Give feedback.
-
|
@gandham-santhosh |
Beta Was this translation helpful? Give feedback.
-
|
@gandham-santhosh can you please update us on this ? |
Beta Was this translation helpful? Give feedback.
-
|
Closing the Discussion we have solved the issue with the knowlg and Inquiry |
Beta Was this translation helpful? Give feedback.
-
we are trying to setup a inquiry on EKS cluster on which we have kafka and neo4j running.
So started to deploy the inquiry components like jobmanager and task manager for question set service on top of it.
we are referring to a git repo : https://github.com/project-sunbird/sunbird-ed-installer/tree/release-6.0.0/helmcharts/inquirybb/charts/flink
When we are deploying the services in EKS we are getting issue in jobmanager for question set republish.
We have gone through the logs so there are some files are missing for the services to run i.e. base config file and log4h properties
files. can we get that ?? or what is the simplest and quick way to do that ?
Issue we are getting :
Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. ... 11 more Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: No configuration setting found for key 'kafka' at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242) ~[flink-dist_2.12-1.13.5.jar:1.13.5] ... 10 more Caused by: com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'kafka' at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:152) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:145) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:172) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:184) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:189) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:246) ~[async-questionset-publish-1.0.0.jar:?] at org.sunbird.job.BaseJobConfig.<init>(BaseJobConfig.scala:16) ~[async-questionset-publish-1.0.0.jar:?] at org.sunbird.job.publish.config.PublishConfig.<init>(PublishConfig.scala:6) ~[async-questionset-publish-1.0.0.jar:?] at org.sunbird.job.questionset.republish.task.QuestionSetRePublishConfig.<init>(QuestionSetRePublishConfig.scala:13) ~[questionset-republish-1.0.0.jar:?] at org.sunbird.job.questionset.republish.task.QuestionSetRePublishStreamTask$.main(QuestionSetRePublishStreamTask.scala:51) ~[questionset-republish-1.0.0.jar:?] at org.sunbird.job.questionset.republish.task.QuestionSetRePublishStreamTask.main(QuestionSetRePublishStreamTask.scala) ~[questionset-republish-1.0.0.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242) ~[flink-dist_2.12-1.13.5.jar:1.13.5] ... 10 more 2024-05-16 11:38:42,968 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Fatal error occurred in the cluster entrypoint. org.apache.flink.util.FlinkException: Application failed unexpectedly. at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAndShutdownClusterAsync$0(ApplicationDispatcherBootstrap.java:170) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.CompletableFuture.uniHandle(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source) ~[?:?] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:257) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.lambda$runApplicationAsync$1(ApplicationDispatcherBootstrap.java:212) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?] at java.util.concurrent.FutureTask.run(Unknown Source) [?:?] at org.apache.flink.runtime.concurrent.akka.ActorSystemScheduledExecutorAdapter$ScheduledFutureTask.run(ActorSystemScheduledExecutorAdapter.java:159) [flink-dist_2.12-1.13.5.jar:1.13.5] at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) [flink-dist_2.12-1.13.5.jar:1.13.5] at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44) [flink-dist_2.12-1.13.5.jar:1.13.5] at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.12-1.13.5.jar:1.13.5] at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.12-1.13.5.jar:1.13.5] at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.12-1.13.5.jar:1.13.5] at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.12-1.13.5.jar:1.13.5] Caused by: java.util.concurrent.CompletionException: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. at java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniCompose.tryFire(Unknown Source) ~[?:?] ... 13 more Caused by: org.apache.flink.client.deployment.application.ApplicationExecutionException: Could not execute application. ... 11 more Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: No configuration setting found for key 'kafka' at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242) ~[flink-dist_2.12-1.13.5.jar:1.13.5] ... 10 more Caused by: com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'kafka' at com.typesafe.config.impl.SimpleConfig.findKeyOrNull(SimpleConfig.java:152) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:145) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.findOrNull(SimpleConfig.java:172) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:184) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:189) ~[async-questionset-publish-1.0.0.jar:?] at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:246) ~[async-questionset-publish-1.0.0.jar:?] at org.sunbird.job.BaseJobConfig.<init>(BaseJobConfig.scala:16) ~[async-questionset-publish-1.0.0.jar:?] at org.sunbird.job.publish.config.PublishConfig.<init>(PublishConfig.scala:6) ~[async-questionset-publish-1.0.0.jar:?] at org.sunbird.job.questionset.republish.task.QuestionSetRePublishConfig.<init>(QuestionSetRePublishConfig.scala:13) ~[questionset-republish-1.0.0.jar:?] at org.sunbird.job.questionset.republish.task.QuestionSetRePublishStreamTask$.main(QuestionSetRePublishStreamTask.scala:51) ~[questionset-republish-1.0.0.jar:?] at org.sunbird.job.questionset.republish.task.QuestionSetRePublishStreamTask.main(QuestionSetRePublishStreamTask.scala) ~[questionset-republish-1.0.0.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?] at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?] at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114) ~[flink-dist_2.12-1.13.5.jar:1.13.5] at org.apache.flink.client.deployment.application.ApplicationDispatcherBootstrap.runApplicationEntryPoint(ApplicationDispatcherBootstrap.java:242) ~[flink-dist_2.12-1.13.5.jar:1.13.5] ... 10 more 2024-05-16 11:38:42,973 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Shutting StandaloneApplicationClusterEntryPoint down with application status UNKNOWN. Diagnostics Cluster entrypoint has been closed externally.. 2024-05-16 11:38:42,977 INFO org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Shutting down rest endpoint. 2024-05-16 11:38:42,985 INFO org.apache.flink.runtime.blob.BlobServer [] - Stopped BLOB server at 0.0.0.0:6124 2024-05-16 11:38:43,069 INFO org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Removing cache directory /tmp/flink-web-d64a1a1b-ae76-4fc1-8713-a7ff06eb6998/flink-web-ui 2024-05-16 11:38:43,069 INFO org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - http://questionset-republish-jobmanager:8081 lost leadership 2024-05-16 11:38:43,070 INFO org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint [] - Shut down complete. 2024-05-16 11:38:43,070 INFO org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent [] - Closing components. 2024-05-16 11:38:43,071 INFO org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunner [] - DefaultDispatcherRunner was revoked the leadership with leader id 00000000-0000-0000-0000-000000000000. Stopping the DispatcherLeaderProcess. 2024-05-16 11:38:43,073 INFO org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess [] - Stopping SessionDispatcherLeaderProcess. 2024-05-16 11:38:43,074 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopping dispatcher akka.tcp://flink@questionset-republish-jobmanager:6123/user/rpc/dispatcher_1. 2024-05-16 11:38:43,075 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopping all currently running jobs of dispatcher akka.tcp://flink@questionset-republish-jobmanager:6123/user/rpc/dispatcher_1. 2024-05-16 11:38:43,077 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Closing the slot manager. 2024-05-16 11:38:43,077 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager [] - Suspending the slot manager. 2024-05-16 11:38:43,078 INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher [] - Stopped dispatcher akka.tcp://flink@questionset-republish-jobmanager:6123/user/rpc/dispatcher_1. 2024-05-16 11:38:43,123 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopping Akka RPC service. 2024-05-16 11:38:43,158 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopping Akka RPC service. 2024-05-16 11:38:43,225 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Shutting down remote daemon. 2024-05-16 11:38:43,227 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remote daemon shut down; proceeding with flushing remote transports. 2024-05-16 11:38:43,253 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Shutting down remote daemon. 2024-05-16 11:38:43,257 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remote daemon shut down; proceeding with flushing remote transports. 2024-05-16 11:38:43,321 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remoting shut down. 2024-05-16 11:38:43,336 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remoting shut down. 2024-05-16 11:38:43,404 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopped Akka RPC service. 2024-05-16 11:38:43,407 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopped Akka RPC service. 2024-05-16 11:38:43,408 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Terminating cluster entrypoint process StandaloneApplicationClusterEntryPoint with exit code 1445.#InquiryCommunityConnect
#108
Beta Was this translation helpful? Give feedback.
All reactions