Two experimental methods, optical limiting and degenerate four-wave mixing, are employed to study the nonlinear optical properties of various novel fullerenes structures. Optical limiting refers to decreased transmittance of a material with increased incident light intensity. Detailed measurements of the wavelength-dependence of fullerene optical limiters have illustrated several key features of reverse saturable absorption. Most important among these is the requirement of weak but non-negligible ground state absorption. We have shown that the optical limiting performance of C₆₀ can be extended into the near infrared range by appropriate modifications of the structure such as higher cage fullerenes or derivatization of the basic C₆₀ molecule. The higher cage fullerene C₇₆ shows improved optical limiting behavior compared to C₆₀, for wavelengths higher than 650 nm, but becomes a weak limiter in the 800 nm range. C₈₄, even at high concentrations in [alpha]-chloronaphthalene, does not reach the good performance of C₆₀, but instead shows weak optical limiting in the 800 nm range.

We also demonstrate that by attaching various groups to the C₆₀ molecule, we can extend the optical limiting performance in the near infrared regime. The C₆₀ derivatives studied, (C₆₀ cyclic ketone, C₆₀ secondary amine, C₆₀CHC₆H₄CO₂H, and C₆₀C₄H₄(CH₃)CH₂O₂C(CH₂)CO₂H), have a similar characteristic: the attached groups cause a symmetry-breaking of the C₆₀ sphere and, therefore, there are new allowed transitions that appear as absorption features up to 750 nm. The optical limiting measurements show that these materials, even for low input energies, have an exceptionally strong optical limiting response in the 640 to 750 nm spectral region. For wavelengths higher than 800 nm, however, they become transparent and no optical limiting is observed. Excited state absorption cross-sections obtained from analysis of the optical limiting data reveal that the C₆₀ derivatives have a maximum triplet-triplet absorption cross-section at 700 nm, which is shifted from the 750 nm value for the C₆₀ molecule. For the first time, optical limiting measurements are performed on five separate C₈₄ isomers. These intriguing results show that the optical limiting behavior is strongly dependent on the cage symmetry. It is also found that the most abundant isomer does not have the strongest optical limiting performance, but is in fact one of the weaker optical limiters of the isomers isolated so far.

The endohedral metallofullerenes are a unique class of fullerene materials and consist of one or more metal atoms encapsulated inside the buckyball cage. An important characteristic of these materials is the charge-transfer from the dopant atoms to the fullerene cage, which has a high electron affinity. The charge-transfer is similar to the optical excitation in a material, but although the electrons are placed in the lowest unoccupied molecular orbital (LUMO), there are no holes produced in the highest occupied molecular orbital (HOMO). This is an important analogy, since it has been previously shown that optical excitation enhances the nonlinear optical properties of a material. The nonresonant degenerate four-wave mixing experiments performed on the endohedral metallofullerene Er₂@C₈₂, at 1064 nm, show that the third order nonlinear susceptibility value is increased by orders of magnitude relative to the empty cage fullerenes, thus, confirming the charge-transfer process from the encapsulated atoms to the fullerene cage. We obtain a value [gamma]_xyyx⁽³⁾( &#173 [omega]; [omega], [omega], &#173 [omega])= &#173 8.65 &#215 10⁻³² esu for the molecular second order hyperpolarizability, which is almost three orders of magnitude larger than the values reported in literature for an empty cage fullerene.