I am having a shell script which opens a file and pass it to python script for processing it. So if there is any issue with the file ( e.g. the file content is not in the format required for the successful execution for python script), the python script throws an exception. Since my objective it to process N files using python script. I need to know which is the file causing the script to break. I read on how to catch the exception in thrown by command execution. . But in my case its the python script which throws the exception and I need to know in shell script what the exception has been thrown. Can anyone help me how can I proceed with this?
Below is the code snippet :
#!/bin/bash yesterday=$(date --date "yesterday" "+%y%m%d") ftoday=$(date --date "today" "+%m-%d-%Y") year=$(date "+%Y") fileList=$(find C:/logdata/$year/$ftoday/ -iname "*"$yesterday".log") for var in $fileList do echo -e "\n START Processing File : $var" >> shelltestlog.txt cat $var| ./scriptA.py echo -e "\n END Processing File : $var" >> shelltestlog.txt done 4 Answers
If your python script returns an non-zero error level whenever it gets an exception, you can use || { } to log messages:
./scriptA.py < "$file" || { printf "\n Python script scriptA.py failed with file \"%s\".\n" "$file" >> shelltestlog.txt } I actually tried to simplify your code first:
#!/bin/bash yesterday=$(date --date "yesterday" "+%y%m%d") ftoday=$(date --date "today" "+%m-%d-%Y") year=$(date "+%Y") readarray -t filesList < <(find C:/logdata/$year/$ftoday/ -iname "*"$yesterday".log") for file in "${filesList[@]}"; do printf "\n START Processing File : %s\n" "$file" >> shelltestlog.txt ./scriptA.py < "$file" || { printf "\n Python script scriptA.py failed with file \"%s\".\n" "$file" >> shelltestlog.txt } printf "\n END Processing File : %s\n" "$file" >> shelltestlog.txt done An uncaught exception will produce a traceback printed to standard error. The best you can do is capture that, and try to parse it.
if ! ./scriptA.py < "$var" 2> stderr.txt; then # Parse stderr.txt to see what exception was raised. fi You should rewrite the shell script in Python and merge it into the existing Python script. The stuff that you are currently doing with the date command, can be done with the datetime and time modules. The stuff you are doing with find can be done with os.walk and fnmatch.fnmatch.
Here's an outline:
#! /usr/bin/python def scriptA(file_to_process): # the code currently in scriptA goes here; # modify it to read from `file_to_process` rather than `sys.stdin` def get_file_list(): # your bash logic to construct the file list goes here, # converted to Python as suggested above def main(): for filename in get_file_list(): sys.stderr.write("Processing {}...\n".format(filename)) try: scriptA(open(filename, "rt")) except SomeExceptionType as e: # handle exception... except SomeOtherExceptionType as e: # handle other kind of exception... sys.stderr.write("Done processing {}.\n".format(filename)) main() You can play some redirection games:
# open file descriptor 3, pointing to the same destination as stdout exec 3>&1 # run the script, # python's stderr goes to stdout so it can be captured in a variable # python's stdout goes to file descriptor 3, displayed on the terminal python_stderr=$( ./scriptA.py < "$var" 2>&1 1>&3 ) # close the file descriptor exec 3>&- Then you can examine $python_stderr for patterns
case $python_stderr in "") echo "nothing on stderr" ;; *foo*) handle_foo_error ;; *bar*) handle_bar_error ;; *baz*) handle_baz_error ;; esac 
