We are orchestrating data pipeline with AWS steps and we do need to run EMR jobs in parallel. I have tried using Map state and it works as expected. The only problem with Map is that in case one step fails , it cancels all the other steps as well. To overcome this issue , I am thinking if we can create an array of steps and pass it dynamically to Branches in parallel state but I have not been able to do it as it is not accepting strings. Is there a workaround for this or can we only hard code branches in Parallel state? Can States.Array() in someway be helpful in this situation?
AWS steps parallel state to orchestrate EMR jobs
272 views Asked by manu At
2
There are 2 answers
0
manu
On
Just for someone who is trying to look for a solution to the stated problem. As suggested by Pooya, I did use catch block inside task within the Map rather than keeping it at map level.The state machine looks like this
Related Questions in AMAZON-EMR
- UIWebView Screen Fitting Issue
- ios responsive design not working (too wide in portrait orientation)
- Setting View orientation to portrait is ignored
- How do I add custom cells to TableView in Swift?
- UIWebView not loading URL in simulator
- What is the limit for number of subscribers to a stream(publisher's) in opentok/tokbox iOS SDK?
- How to generate request format for WCF web service method for Mac and iPhone
- Difference between gethostname() and [NSProcessInfo hostName]?
- How to force close ipad/iphone keypad when input element is not focused using JS?
- iOS app rejected because of in-app purchase
Related Questions in AWS-STEP-FUNCTIONS
- UIWebView Screen Fitting Issue
- ios responsive design not working (too wide in portrait orientation)
- Setting View orientation to portrait is ignored
- How do I add custom cells to TableView in Swift?
- UIWebView not loading URL in simulator
- What is the limit for number of subscribers to a stream(publisher's) in opentok/tokbox iOS SDK?
- How to generate request format for WCF web service method for Mac and iPhone
- Difference between gethostname() and [NSProcessInfo hostName]?
- How to force close ipad/iphone keypad when input element is not focused using JS?
- iOS app rejected because of in-app purchase
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Wrap the inner state machine in a one-branch parallel state and add error/retry policies to it. Basically, you want to catch all errors and ensure that the iteration always succeeds.