Spark and NegativeArraySizeException

Recently I was debugging the following crash in Spark:

Disabling Kryo solves the issue. To do that just set spark.serializer to org.apache.spark.serializer.JavaSerializer. Other workaround is to change Kryo’s reference management, as explained on Github:

Spark and NullPointerException in UTF8String.contains

Recently I was debugging a NullPointerException in Spark. The stacktrace was indicating this:

After some digging I found out that the following query causes the problem:

If I commented out the line with the comment the NPE was no longer there.…

Playing With args4j Part 5 — Safe setters

This is the fifth part of the Playing With args4j series. For your convenience you can find other parts in the table of contents in Part 1 – Mixins There is one more problem with our setters — they are not type safe.…

Playing With args4j Part 4 — Nicer setters

This is the fourth part of the Playing With args4j series. For your convenience you can find other parts in the table of contents in Part 1 – Mixins Last time we ended with the setter of a form

. What we don’t…

Playing With args4j Part 3 — Nice setters

This is the third part of the Playing With args4j series. For your convenience you can find other parts in the table of contents in Part 1 – Mixins Let’s see how we can implement the setters. Idea is as following: we will…

Playing With args4j Part 2 — Automatic getters

This is the second part of the Playing With args4j series. For your convenience you can find other parts in the table of contents in Part 1 – Mixins Last time we created mixins to be able to compose classes with multiple inherited…

Playing With args4j Part 1 — Mixins

This is the first part of the Playing With args4j series. For your convenience you can find other parts using the links below (or by guessing the address): Part 1 — Mixins Part 2 — Automatic getters Part 3 — Nice setters Part…

.NET Internals Cookbook Part 12 — Memory structure, attributes, handles

This is the twelfth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 82. What is a bit-mapped attribute? Custom attribute? Pseudo-custom attribute? There are…

.NET Internals Cookbook Part 11 — Various C# riddles

This is the eleventh part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 75. What is the TimeSpan resolution? It’s 100 nanoseconds, as specified by…

.NET Internals Cookbook Part 10 — Threads, Tasks, asynchronous code and others

This is the tenth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 65. How can you await async void method or catch exceptions thrown…

.NET Internals Cookbook Part 9 — Finalizers, queues, card tables and other GC stuff

This is the ninth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 59. What happens if an exception is thrown in finalizer? What about…

.NET Internals Cookbook Part 8 — C# gotchas

This is the eighth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 52. Can you have class in a class? Class in an interface?…

.NET Internals Cookbook Part 7 — Word tearing, locking and others

This is the seventh part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 45. Should we always avoid boxing? Boxing is expensive, it wraps value…

.NET Internals Cookbook Part 6 — Object internals

This is the sixth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 36. What is a shim? .NET applications are started directly, the same…

.NET Internals Cookbook Part 5 — Methods, parameters, modifiers

This is the fifth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 27. Can two methods differ in return type only? Not in C#,…

.NET Internals Cookbook Part 4 — Type members

This is the fourth part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 18. Can you add type constructor to the interface? In C# no.…

.NET Internals Cookbook Part 3 — Initialization tricks

This is the third part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 13. Can a variable be true and false at the same time?…

.NET Internals Cookbook Part 2 — GC-related things

This is the second part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 7. How to compact the LOH? You need to set GCSettings.LargeObjectHeapCompactionMode property…

.NET Internals Cookbook Part 1 — Exceptions, filters and corrupted processes

This is the first part of the .NET Internals Cookbook series. For your convenience you can find other parts in the table of contents in Part 0 – Table of contents 1. What happens when you throw something which does not inherit from…

.NET Internals Cookbook Part 0 — Table of contents

This is the table of contents of the .NET Internals Cookbook. Look below for links to all answers. In this series I answer various .NET questions. Some of them are asked during interviews, some of them I see on the internet, some of…