Skip to main content
Duke University Libraries
DukeSpace Scholarship by Duke Authors
  • Login
  • Ask
  • Menu
  • Login
  • Ask a Librarian
  • Search & Find
  • Using the Library
  • Research Support
  • Course Support
  • Libraries
  • About
View Item 
  •   DukeSpace
  • Duke Scholarly Works
  • Scholarly Articles
  • View Item
  •   DukeSpace
  • Duke Scholarly Works
  • Scholarly Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Expression of Fractals Through Neural Network Functions

Thumbnail
View / Download
594.1 Kb
Authors
Dym, Nadav
Sober, Barak
Daubechies, Ingrid
Repository Usage Stats
82
views
30
downloads
Abstract
To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that $\ell$ can grow exponentially with the number of network parameters $p$, a property often used to explain the advantages of DNNs over shallow NNs in approximating complicated functions. Nonetheless, a simple dimension argument shows that DNNs cannot generate all piecewise linear functions with $\ell$ linear regions as soon as $\ell > p$. It is thus natural to seek to characterize specific families of functions with $\ell$ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) generate sequences of piecewise linear functions $F_k$ with a number of linear regions exponential in $k$. We show that, under mild assumptions, $F_k$ can be generated by a NN using only $\mathcal{O}(k)$ parameters. IFS are used extensively to generate, at low computational cost, natural-looking landscape textures in artificial images. They have also been proposed for compression of natural images, albeit with less commercial success. The surprisingly good performance of this fractal-based compression suggests that our visual system may lock in, to some extent, on self-similarities in images. The combination of this phenomenon with the capacity, demonstrated here, of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks, as well as suggest new algorithms for representing self similarities in images based on the DNN mechanism.
Type
Journal article
Subject
cs.LG
cs.LG
cs.IT
math.IT
stat.ML
Permalink
https://hdl.handle.net/10161/19596
Collections
  • Scholarly Articles
More Info
Show full item record

Scholars@Duke

Dym

Nadav Dym

Phillip Griffiths Assistant Research Professor
Sober

Barak Sober

Phillip Griffiths Assistant Research Professor
I am currently privilaged to be working with Prof. Ingrid Daubechies. Before that, I have completed my PhD in applied mathematics at Tel-Aviv University under the mentoring of Prof. David Levin. My MSc was co-mentored by Prof. Levin and Prof. Israel Finkelstein from the Department of Archaeology and Ancient Near Eastern Civilizations. My research ranges between analysis of high dimensional data from a geometrical perspective and the applicatio
Alphabetical list of authors with Scholars@Duke profiles.
Open Access

Articles written by Duke faculty are made available through the campus open access policy. For more information see: Duke Open Access Policy

Rights for Collection: Scholarly Articles


Works are deposited here by their authors, and represent their research and opinions, not that of Duke University. Some materials and descriptions may include offensive content. More info

Make Your Work Available Here

How to Deposit

Browse

All of DukeSpaceCommunities & CollectionsAuthorsTitlesTypesBy Issue DateDepartmentsAffiliations of Duke Author(s)SubjectsBy Submit DateThis CollectionAuthorsTitlesTypesBy Issue DateDepartmentsAffiliations of Duke Author(s)SubjectsBy Submit Date

My Account

LoginRegister

Statistics

View Usage Statistics
Duke University Libraries

Contact Us

411 Chapel Drive
Durham, NC 27708
(919) 660-5870
Perkins Library Service Desk

Digital Repositories at Duke

  • Report a problem with the repositories
  • About digital repositories at Duke
  • Accessibility Policy
  • Deaccession and DMCA Takedown Policy

TwitterFacebookYouTubeFlickrInstagramBlogs

Sign Up for Our Newsletter
  • Re-use & Attribution / Privacy
  • Harmful Language Statement
  • Support the Libraries
Duke University