Monthly Archives: March 2020

Archive of posts published in the specified Month

Running Anaconda with DGL and mxnet on CUDA GPU in Spark running in EMR

Written on Mar, 28, 2020

Today I’m going to share my configuration for running custom Anaconda Python with DGL (Deep Graph Library) and mxnet library, with GPU support via CUDA, running in Spark hosted in EMR. Actually, I have Redshift configuration as well, with support for gensim, tensorflow,…

Read more

Running any query in Redshift or JDBC from Spark in EMR

Written on Mar, 21, 2020

Last time we saw how to connect to Redshift from Spark running in EMR. Provided solution was nice but allowed for reading data only. Sometimes we might want to run any DDL or DML query, not only simple read statements. To do that,…

Read more

Connecting to Redshift from Spark running in EMR

Written on Mar, 14, 2020

Today I’ll share my configuration for Spark running in EMR to connect to Redshift cluster. First, I assume the cluster is accessible (so configure virtual subnet, allowed IPs and all network stuff before running this). I’m using Zeppelin so I’ll show two interpreters…

Read more

UpdateOrchestrator waking up the machine

Written on Mar, 07, 2020

Recently I was investigating machine waking up from sleep without any user interaction. It was a bit irritating because user was putting his PC to sleep and leaving it for a day just to notice in the evening that the machine woke up…

Read more