Getting error while running query on hive over tez -

-

getting error while running query on hive on tez. per logs, hive failing while copying tez jars hdfs location on start of tez session.below complete log obtained hive log file :

2015-06-19 01:23:52,289 info  [hiveserver2-background-pool: thread-41]: ql.driver (sessionstate.java:printinfo(852)) - query id = saurabh_20150619012323_f52f1d6c-2adb-4edc-8ba4-b64d7d898325 2015-06-19 01:23:52,289 info  [hiveserver2-background-pool: thread-41]: ql.driver (sessionstate.java:printinfo(852)) - total jobs = 1 2015-06-19 01:23:52,289 info  [hiveserver2-background-pool: thread-41]: log.perflogger (perflogger.java:perflogend(148)) - </perflog method=timetosubmit start=1434657232288 end=1434657232289 duration=1 from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:23:52,290 info  [hiveserver2-background-pool: thread-41]: log.perflogger (perflogger.java:perflogbegin(121)) - <perflog method=runtasks from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:23:52,290 info  [hiveserver2-background-pool: thread-41]: log.perflogger (perflogger.java:perflogbegin(121)) - <perflog method=task.tez.stage-1 from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:23:52,302 info  [hiveserver2-background-pool: thread-41]: ql.driver (sessionstate.java:printinfo(852)) - launching job 1 out of 1 2015-06-19 01:23:52,302 info  [hiveserver2-background-pool: thread-41]: ql.driver (driver.java:launchtask(1630)) - starting task [stage-1:mapred] in parallel 2015-06-19 01:23:52,312 info  [thread-21]: session.sessionstate (sessionstate.java:start(488)) - no tez session required @ point. hive.execution.engine=mr. 2015-06-19 01:23:52,314 info  [thread-21]: tez.tezsessionpoolmanager (tezsessionpoolmanager.java:getsession(125)) - queuename: null nondefaultuser: true defaultqueuepool: null blockingqueuelength: -1 2015-06-19 01:23:52,315 info  [thread-21]: tez.tezsessionpoolmanager (tezsessionpoolmanager.java:getnewsessionstate(154)) - created new session queue: null session id: 85d83746-a48e-419e-a7ca-8c98faf173ea 2015-06-19 01:23:52,380 info  [thread-21]: configuration.deprecation (configuration.java:warnonceifdeprecated(1049)) - mapred.committer.job.setup.cleanup.needed deprecated. instead, use mapreduce.job.committer.setup.cleanup.needed 2015-06-19 01:23:52,412 info  [thread-21]: ql.context (context.java:getmrscratchdir(328)) - new scratch dir hdfs://localhost:9000/tmp/hive/saurabh/e5a701ae-242d-488f-beec-cf18878becdc/hive_2015-06-19_01-23-49_794_2167174123575230985-2 2015-06-19 01:23:52,420 info  [thread-21]: exec.task (teztask.java:updatesession(233)) - tez session hasn't been created yet. opening session 2015-06-19 01:23:52,420 info  [thread-21]: tez.tezsessionstate (tezsessionstate.java:open(142)) - user of session id 85d83746-a48e-419e-a7ca-8c98faf173ea saurabh 2015-06-19 01:23:52,433 info  [thread-21]: tez.dagutils (dagutils.java:localizeresource(950)) - localizing resource because not exist: file:/usr/lib/tez/* dest: hdfs://localhost:9000/tmp/hive/saurabh/_tez_session_dir/85d83746-a48e-419e-a7ca-8c98faf173ea/* 2015-06-19 01:23:52,433 info  [thread-21]: tez.dagutils (dagutils.java:localizeresource(954)) - looks thread writing same file wait. 2015-06-19 01:23:52,433 info  [thread-21]: tez.dagutils (dagutils.java:localizeresource(961)) - number of wait attempts: 5. wait interval: 5000 2015-06-19 01:24:17,449 error [thread-21]: tez.dagutils (dagutils.java:localizeresource(977)) - not find jar being uploaded 2015-06-19 01:24:17,451 error [thread-21]: exec.task (teztask.java:execute(184)) - failed execute tez graph. java.io.ioexception: previous writer failed write hdfs://localhost:9000/tmp/hive/saurabh/_tez_session_dir/85d83746-a48e-419e-a7ca-8c98faf173ea/*. failing because unlikely write too.     @ org.apache.hadoop.hive.ql.exec.tez.dagutils.localizeresource(dagutils.java:978)     @ org.apache.hadoop.hive.ql.exec.tez.dagutils.addtempresources(dagutils.java:859)     @ org.apache.hadoop.hive.ql.exec.tez.dagutils.localizetempfilesfromconf(dagutils.java:802)     @ org.apache.hadoop.hive.ql.exec.tez.tezsessionstate.refreshlocalresourcesfromconf(tezsessionstate.java:228)     @ org.apache.hadoop.hive.ql.exec.tez.tezsessionstate.open(tezsessionstate.java:154)     @ org.apache.hadoop.hive.ql.exec.tez.teztask.updatesession(teztask.java:234)     @ org.apache.hadoop.hive.ql.exec.tez.teztask.execute(teztask.java:136)     @ org.apache.hadoop.hive.ql.exec.task.executetask(task.java:160)     @ org.apache.hadoop.hive.ql.exec.taskrunner.runsequential(taskrunner.java:88)     @ org.apache.hadoop.hive.ql.exec.taskrunner.run(taskrunner.java:75) 2015-06-19 01:24:18,329 error [hiveserver2-background-pool: thread-41]: ql.driver (sessionstate.java:printerror(861)) - failed: execution error, return code 1 org.apache.hadoop.hive.ql.exec.tez.teztask 2015-06-19 01:24:18,329 info  [hiveserver2-background-pool: thread-41]: log.perflogger (perflogger.java:perflogend(148)) - </perflog method=driver.execute start=1434657232288 end=1434657258329 duration=26041 from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:24:18,329 info  [hiveserver2-background-pool: thread-41]: log.perflogger (perflogger.java:perflogbegin(121)) - <perflog method=releaselocks from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:24:18,329 info  [hiveserver2-background-pool: thread-41]: log.perflogger (perflogger.java:perflogend(148)) - </perflog method=releaselocks start=1434657258329 end=1434657258329 duration=0 from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:24:18,333 error [hiveserver2-background-pool: thread-41]: operation.operation (sqloperation.java:run(200)) - error running hive query:  org.apache.hive.service.cli.hivesqlexception: error while processing statement: failed: execution error, return code 1 org.apache.hadoop.hive.ql.exec.tez.teztask     @ org.apache.hive.service.cli.operation.operation.tosqlexception(operation.java:315)     @ org.apache.hive.service.cli.operation.sqloperation.runquery(sqloperation.java:147)     @ org.apache.hive.service.cli.operation.sqloperation.access$100(sqloperation.java:70)     @ org.apache.hive.service.cli.operation.sqloperation$1$1.run(sqloperation.java:197)     @ java.security.accesscontroller.doprivileged(native method)     @ javax.security.auth.subject.doas(subject.java:415)     @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1628)     @ org.apache.hive.service.cli.operation.sqloperation$1.run(sqloperation.java:209)     @ java.util.concurrent.executors$runnableadapter.call(executors.java:471)     @ java.util.concurrent.futuretask.run(futuretask.java:262)     @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145)     @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615)     @ java.lang.thread.run(thread.java:745) 2015-06-19 01:24:18,342 info  [hiveserver2-handler-pool: thread-29]: exec.listsinkoperator (operator.java:close(595)) - 40 finished. closing...  2015-06-19 01:24:18,343 info  [hiveserver2-handler-pool: thread-29]: exec.listsinkoperator (operator.java:close(613)) - 40 close done 2015-06-19 01:24:18,393 info  [hiveserver2-handler-pool: thread-29]: log.perflogger (perflogger.java:perflogbegin(121)) - <perflog method=releaselocks from=org.apache.hadoop.hive.ql.driver> 2015-06-19 01:24:18,394 info  [hiveserver2-handler-pool: thread-29]: log.perflogger (perflogger.java:perflogend(148)) - </perflog method=releaselocks start=1434657258393 end=1434657258394 duration=1 from=org.apache.hadoop.hive.ql.driver>


Comments

Post a Comment

Popular posts from this blog

PHP DOM loadHTML() method unusual warning -

-
i had following code works fine on localhost: $document = new domdocument(); $document->loadhtml($p_result); $form = $document->getelementsbytagname('form')->item(0); // code continues using $form variable after same code updated on outside server, loadhtml() failed , issued warning. warning: domdocument::loadhtml() expects parameter 1 valid path, string given in path/name/to/script.php returned null instead of object code pretty gets fatal error. note contents of $p_result absolutely same on outside server , on localhost. but why displays kind of warning , why not work? doesn't loadhtml() expects argument 1 string in first place? why method expects parameter 1 valid path ? just make clear i'm not calling loadhtmlfile() , i'm calling loadhtml() . thanks. you're affected one of php bugs . issue present in php 5.6.8 , 5.6.9. have affected php version on server, , bug-free version on localhost. the bug forbids null
Read more

python - How to create jsonb index using GIN on SQLAlchemy? -

-
here's current code creating index jsonb. index("mytable_data_idx_id_key", mytable.data['id'].astext, postgresql_using='gin') but got error. sqlalchemy.exc.programmingerror: (psycopg2.programmingerror) data type text has no default operator class access method "gin" hint: must specify operator class index or define default operator class data type. [sql: "create index event_data_idx_id_key on event using gin ((data ->> 'id'))"] is there way create index on sqlalchemy? the postgresql specific sqlalchemy docs @ http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#operator-classes mention postgresql_ops dictionary provide "operator class" used postgresql, , provide example illustrating use: index('my_index', my_table.c.id, my_table.c.data, postgresql_ops={ 'data': 'text_pattern_ops',
Read more

c# - TransactionScope not rolling back although no complete() is called -

-
i'm using transactionscope rollback transaction fail bool errorreported = false; action<importerrorlog> newerrorcallback = e => { errorreported = true; errorcallback(e); }; using (var transaction = new transactionscope()) { foreach (importtaskdefinition task in taskdefinition) { loader.load(streamfile, newerrorcallback, task.destinationtable, processingtaskid); } if (!errorreported) transaction.complete(); } i'm sure there no transactionscope started ahead or after code. i'm using entity framework insert in db. regardless state of errorreported transaction never rolled in case of error. what missing ? transactionscope sets transaction.current . that's does. wants transacted must @ property. i believe ef each time connection opened reason. probably, connection open when scope installed. open connection inside of scope or enlist manually. ef has nasty "design decision": default opens
Read more