ubuntu - How to view the terminal log when executing a remote Apache Pig script -


i have written script in apache pig being executed on hadoop cluster. loads ~55 thousand of .gz files , takes 35 hours complete. should store results in file, didn't, want debug happened.

i execute script typing:

pig script.pig

in terminal of ubuntu virtual machine see information messages progress of job , on. how can view these messages when shutting down local machine? need keep local machine running full 35 hours see these messages?

i know can type command in ubuntu , store response typing:

command > logfile.txt

but think won't work here, since connection between hadoop hdfs system , virtual machine lost during shutting down local machine, log messages must stored on hadoop cluster. how achieve this, or done automatically , don't know look?

if run local machine if process gets killed (e.g. shutting down machine) hadoop kill job. that's why you'd want start long running jobs machine in cluster or portal node runs 24/7.

if available you, can use oozie run job, oozie server take care of running job , provide logs.


Comments

Post a Comment

Popular posts from this blog

How to provide Authorization & Authentication using Asp.net, C#? -

Image
i have asp.net application in c#. i have provide page authorization specific user/usergroup. i have created 2 tables in database: 1) pagename 2)permissions[insert, update, view, delete] but i'm unable match tables , permissions assigning... can 1 please me, how created tables in database , how coding. please me. you can below explanation. configure access specific file , folder, set forms-based authentication. request page in application redirected logon.aspx automatically. in web.config file, type or paste following code. this code grants users access default1.aspx page , subdir1 folder . <configuration> <system.web> <authentication mode="forms" > <forms loginurl="login.aspx" name=".aspnetauth" protection="none" path="/" timeout="20" > </forms> </authentication> <!-- section denies access files in...
Read more

toolbar - How to add link to user registration inside toobar in admin joomla 3 custom component -

im trying write component admin side , need add user creation. dont want duplicate existing codes add link. know how add task not simple link toolbar. thx help. if wanting button opens new user screen can try in component: $bar = jtoolbar::getinstance('toolbar'); $bar->appendbutton('link', 'users', 'new user','index.php?option=com_users&task=user.add'); you need add view.html.php file wherever you're creating toolbar. is need?
Read more

How to use Authorization & Authentication in Asp.net, C#? -

i using roll management , trying give page , folder access according user or user group, using server created ad group user authentication. i have default1.aspx page default , subdir1 folder give different access separate user group i using below logic in web.config. <location path="subdir1"> <system.web> <authorization> <allow users ="?" /> </authorization> </system.web> </location> i facing problem provide same access 2 or more directory same user should have provide allow user code twice both folder? i can use logic repeating value folder want access providing in 1 logic. i have got answer configure folder/page access, have make different access shown below.. configure access specific file , folder, set forms-based authentication. request page in application redirected logon.aspx automatically. in web.config file, done following code. this code grants user...
Read more