Today, I tried the glm function from the SparkR package. And it seemed to have initially solved the problem with the glm function. However, when you save the result of the glm function in a variable, things seem to go wrong. But only when the variable explorer is open. When it refreshes, things go wrong, and you can't run anything anymore. If you keep the variable explorer closed, there seems to be no issue. So, I suspect that the variable explorer has trouble loading the glm object or something, and everything hangs on that. This object is generally quite large.