No figure output

[rb643]

I appear to have lost the ability to plot image, when I run the tutorial code below it does not produce any image (either in the viewer window nor does it open a new window)... I've updated the installation, updated rgl, checked that my Quartz is up to date and nothing seems to make much difference... I'm running R version 4.0.3 on a Mac...

subjects_dir = fsbrain::get_optional_data_filepath("subjects_dir");
subject_id = 'subject1';
surface = 'white';
hemi = 'both';
label = 'cortex.label';
vis.subject.label(subjects_dir, subject_id, label, hemi, views="si")
vis.subject.annot(subjects_dir, 'subject1', 'aparc', 'both', views=c('si'));

In addition, when I try and run vis.export.from.coloredmeshes with some of my own data I get the following error:

Error in magick_image_trim(image, fuzz) : 
  rsession: GeometryDoesNotContainImage `/private/var/folders/mg/mtwws03n2492cyzkbphbvt5m0000gn/T/Rtmp5TvJWr/sd_lateral_lhd98779fe97c9.png' @ warning/attribute.c/GetImageBoundingBox/532

I suspect these are somehow related problems, but no clue where to even start troubleshooting...

[dfsp-spirit]

I think this is a downstream problem with the rgl package. Could you try plotting a simple example figure, e.g. (taken from here:

library(rgl)
data(iris)
x <- sep.l <- iris$Sepal.Length
y <- pet.l <- iris$Petal.Length
z <- sep.w <- iris$Sepal.Width
rgl.open() # Open a new RGL device
rgl.points(x, y, z, color ="lightgray") # Scatter plot

Does that give you a plot?

Have you seen #45? It is just a wild guess, but maybe the solution given there (calling library(rgl); rglwidget(); after plotting) also helps?

Apart from that, do you remember what you changed on your system before you lost the ability to plot images? Did you update MacOS, or something else? Which MacOS version and Xquartz version are you running? The last one I had access to (and could confirm that fsbrain works on it), was Big Sur. I remember that I had to update Xquartz at some time to keep rgl working under Big Sur.

Also see #47 for some background.

[rb643]

I can run the example code but indeed need to explicitly call the rglwidget to see any output. I'm running the latest mac OS (ventura) with the latest xquarx (and can run the display command fine too)... I'm creating a bunch of figures looping through various FreeSurfer output folder using this command:

  coloredmeshes = vis.symmetric.data.on.subject(subjects_dir,  subject_id, surface = "inflated",
                                            lh_thresh, rh_thresh, bg='curv_light', rglactions=list( 'no_vis'=T),
                                              makecmap_options = list('colFn'=colmap,'range'=c(-6, 6))) #,'range'=c(-5, 5)
  img = vis.export.from.coloredmeshes(coloredmeshes,
                                   colorbar_legend=paste0(measure,": (t) - ", cc, " - s",ss),
                                     output_img=paste0('./Figures/FreeSurfer_group/',measure,'_',cc,'_FWMH',ss,'.png'))

where the measure, cc and ss variables are iterated over to denote the phenotypes, group comparison and smoothness level and the maps are thresholded based on their relative significance. Sadly adding an rglwidget() command before either of these two command didn't fix anything as I still get the same error.

Last time this worked was indeed on Big Sur, but first time the error message popped up I was on Montery (in fact I upgraded so I could also update xcode and xquartz in the hope that that would solve this... Sadly it hasn't

[dfsp-spirit]

I will ask around friends to see whether I can get someone with a Mac to install fsbrain and check this out, I guess.

Trying to get this sorted out via some CI service running Mac OS instances does not sound like a good idea.

[dfsp-spirit]

Hi Richard (@rb643),

I investigated this further and have 2 things:

1. To check whether rgl is working, could you run this:

library(rgl)
demo(rgl)

Then press returns a couple of times to watch the demos. Do they show up?

2. To try to solve your problem, could you try the following before calling your plots:

options(rgl.printRglwidget = TRUE)

[rb643]

Thanks Tim! The demo works fine (with the option enabled), but when printing to a file as in the code above I still get the same error:

Error in magick_image_trim(image, fuzz) : 
  rsession: GeometryDoesNotContainImage `/private/var/folders/mg/mtwws03n2492cyzkbphbvt5m0000gn/T/RtmpSoHU2m/sd_lateral_lh10c2248c1d1ac.png' @ warning/attribute.c/GetImageBoundingBox/532

[dmurdoch]

I am the rgl maintainer. I can't run the code in the first comment; it dies here

> library(fsbrain)
> subjects_dir = fsbrain::get_optional_data_filepath("subjects_dir");
Error in pkgfilecache::get_filepath(pkg_info, filename, mustWork = mustWork) : 
  File 'subjects_dir' (from 'subjects_dir') does not exist in local package cache at '~/Library/Application Support/fsbrain/subjects_dir', and mustWork is TRUE.

before it ever gets to any rgl code. After applying some patches (PR coming as soon as the checks finish), I get displays during checking. I don't have any code that produces a display other than the checks, and they take forever to run, so I don't know if things worked before my patches.

[rb643]

will give this a try once the patch is there, thanks for looking into it! The "subjects_dir" is the default subjects directory that comes with the FreeSurfer installation in my case I specificy that as a global outside this code subjects_dir = '/Applications/freesurfer/7.3.2/subjects/' then same with subject_id I use the fsaverage from FreeSurfer subject_id = 'fsaverage';. the rh_thresh and lh_tresh are vectors that have the values the length of the number of vertices in that fsaverage template.

[dfsp-spirit]

If the demo runs for Richard, then this is indeed not an rgl problem. Thanks a lot for still looking into it, @dmurdoch.

[dmurdoch]

I don't know what FreeSurfer is, but perhaps it is interfering with rgl somehow. A very simple check would be to run rgl::rgl.useNULL() in your session. If that returns TRUE, then someone (FreeSurfer? a leftover environment variable? some R script?) has asked rgl not to use OpenGL, so it won't display anything in R, but rglwidget() should still display things in a browser.

[rb643]

This might be the crux then cause that is indeed what I get, though it seems to be related to the build I am using?

> rgl::rgl.useNULL()
[1] TRUE
> library(rgl)
This build of rgl does not include OpenGL functions.  Use
 rglwidget() to display results, e.g. via options(rgl.printRglwidget = TRUE).

My xquartz is up to date so is there a way to (re-)build this with OpenGL support that I am missing?

[rb643]

After reinstalling rgl from the repo, reinstalling quartz and updating pretty much all R possible dependencies I still have the same error and loading the rgl library still tells me the build doesn't support openGL, however when running the original code I now also get a number of additional warning so leaving those here in case they are relevant:

Error in magick_image_trim(image, fuzz) : 
  rsession: GeometryDoesNotContainImage `/private/var/folders/mg/mtwws03n2492cyzkbphbvt5m0000gn/T/RtmpGb9eJ4/sd_lateral_lhd5183412c630.png' @ warning/attribute.c/GetImageBoundingBox/532
In addition: Warning messages:
1: In min(x, na.rm = TRUE) :
  no non-missing arguments to min; returning Inf
2: In max(x, na.rm = TRUE) :
  no non-missing arguments to max; returning -Inf
3: 'rgl::rgl.viewpoint' is deprecated.
Use 'view3d' instead.
See help("Deprecated") 
4: In rgl::rgl.snapshot(output_image, fmt = "png") :
  this build of rgl does not support snapshots
5: 'rgl::rgl.viewpoint' is deprecated.
Use 'view3d' instead.
See help("Deprecated") 
6: In rgl::rgl.snapshot(output_image, fmt = "png") :
  this build of rgl does not support snapshots
7: 'rgl::rgl.viewpoint' is deprecated.
Use 'view3d' instead.
See help("Deprecated") 
8: In rgl::rgl.snapshot(output_image, fmt = "png") :
  this build of rgl does not support snapshots
9: 'rgl::rgl.viewpoint' is deprecated.
Use 'view3d' instead.
See help("Deprecated") 
10: In rgl::rgl.snapshot(output_image, fmt = "png") :
  this build of rgl does not support snapshots

[dmurdoch]

The deprecation warnings are coming from the devel version of rgl. You won't get those from the CRAN build, and it should also support OpenGL.

If you really want to use the devel version, you need to have several development libraries available on your system. I get include files from these locations:

• /opt/X11/include
• /usr/local/opt/freetype/include/freetype2

There are also these includes that are needed even without OpenGL, so I think you have them (or equivalents):

• /usr/local/include/libpng16
• /usr/local/include

I forget now where I found the X11 and freetype2 includes. It's possibly via brew or by following the instructions here: https://github.com/R-macos/recipes. But I'd recommend using the CRAN build for now, or you're going to see a lot of deprecation warnings.

[rb643]

That makes sense. With the Cran install I still get the warning that the build doesn't support OpenGL, am I missing anything to force that support?

[dmurdoch]

That's not good. I'm on an Intel Mac, and the CRAN build is fine. Presumably you're on an M1 or M2? Can you show me what rgl prints in a clean session when you just run library(rgl)?

[rb643]

I'm actually still on the intell (i9) mac, in a fresh Rstudio:

> library(rgl)
This build of rgl does not include OpenGL functions.  Use
 rglwidget() to display results, e.g. via options(rgl.printRglwidget = TRUE).
> version
               _                           
platform       x86_64-apple-darwin17.0     
arch           x86_64                      
os             darwin17.0                  
system         x86_64, darwin17.0          
status                                     
major          4                           
minor          0.3                         
year           2020                        
month          10                          
day            10                          
svn rev        79318                       
language       R                           
version.string R version 4.0.3 (2020-10-10)
nickname       Bunny-Wunnies Freak Out    

Running Mac Ventura 13.0.1. (and XQuartz 2.8.4).

[dmurdoch]

That's a somewhat old version of R, and a very new version of macOS. I would guess that Ventura has moved the locations of things, and CRAN is not updating old binaries. So I'd suggest updating to current R (4.2.2), and in the meantime I'll get in touch with CRAN to see if they have any other advice. Be aware that when you update R all existing packages will need to be reinstalled, so it might be time consuming.

[rb643]

On it. And will keep you posted if that solves things.

[rb643]

Seems that did indeed fix it. My apologies this seems to have been very much in the "have you tried turning it on and off again" class of problems.

[dmurdoch]

Staying up to date with R releases helps a lot. They only distribute binaries for 4.1.x and 4.2.x now, and will drop 4.1.x in a few months. So this update will make package installs a lot faster and easier for you.

[dfsp-spirit]

Great to hear that this could finally be solved.