Why do I keep seeing "File name too long" within my cells?

Because Zeppelin Notebooks generate dynamic Java/Scala inner classes for each cell that is run, the names of these cells - particularly the more the notebook cell is re-run during development/debugging scenarios - can get quite large and exceed the underlying filesystem maximium file length (ie. 255 chars).

The short-term workaround is to restart the interpreter as this will restart the inner class naming mechanism and buy you more time.

The long-term workaround requires engineering work to avoid this situation altogether.  This is relatively low-level work as the current notebooks (including Databricks' own notebook which also has this problem) relies on the Spark REPL which relies on the Scala REPL.

The underlying work would need to be done at these REPL layers which is not a light task.  Further complicating things:  the Scala REPL has changed quite a bit between Scala 2.10 and Scala 2.11.

Have more questions? Submit a request

Comments

  • Avatar
    pkmarko sarnych

    You can use a third party software to bypass long path errors. I’d recommend you to try Long Path Tool. It was very helpful for me.