"pandas.errors.EmptyDataError: No columns to parse from file" error from stable diffusion

 Stable Diffusion Error

I ran into this issue after the disk on which I was running stable diffusion AI image generator ran out of disk space.  0 bytes free.  Somehow the headings row of txt2img_logs.csv was removed, and the script wouldn't run.

Traceback (most recent call last):
  File "optimizedSD/optimized_txt2img.py", line 188, in <module>
    logger(vars(opt), log_csv = "logs/txt2img_logs.csv")
  File "E:\Projects\stable-diffusion\optimizedSD\optimUtils.py", line 57, in logger
    df = pd.read_csv(log_csv)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\util\_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\util\_decorators.py", line 317, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 605, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 1442, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 1747, in _make_engine
    return mapping[engine](f, **self.options)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 92, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas\_libs\parsers.pyx", line 554, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

The solution was to simply remove the log file which allowed the script to re-create it.

The headings are the following in case you need to recreate it for some reason:

prompt,    outdir,    skip_grid,    skip_save,    ddim_steps,    fixed_code,    ddim_eta,    n_iter,    H,    W,    C,    f,    n_samples,    n_rows,    scale,    device,    from_file,    seed,    unet_bs,    turbo,    precision,    format,    sampler,    ckpt

It's worth noting that I'm actually running OptimizedSD as it works well  (albeit slower) with less video ram.  There's some chance the log format is changed. 

WSUS.msc RSAT administration tool missing after Windows 10 upgrade

 After upgrading my laptop's version of Windows 10 form 1803 to 1809 recently I discovered that the WSUS administration tool from RSAT was missing.  The .msc file was actually still present but it wouldn't run.  

I considered downloading RSAT again, since that's how I had initially installed the various RSAT tools a few years ago, but found on this Microsoft page new instructions:

"Starting with Windows 10 October 2018 Update, RSAT is included as a set of "Features on Demand" right from Windows 10. Do not download an RSAT package from this page. Instead, just go to "Manage optional features" in Settings and click "Add a feature" to see the list of available RSAT tools. Select and install the specific RSAT tools you need. To see installation progress, click the Back button to view status on the "Manage optional features" page."

Seems easy enough, but it didn't work.  The install failed every time with the error message: "Install failed. Contact your administrator to get this feature."

 I searched for the error message and found this helpful blog post with the following instructions:

1. Disable WSUS by setting HKLM/Software/Policies/Microsoft/Windows/WindowsUpdate/AU/ to 0
2. Restart the Windows Update service
3. Install again

This article doesn't mention reverting the registry change, but I think that's probably warranted. Now it's back to mucking with WSUS!

UPDATE: This appears to affect other RSAT components as well.  The Group Policy Management Console gpmc.msc was also missing.  The same fix applied, so this isn't exclusive to the WSUS RSAT component.

 

Migrate a Centos 7 VM from VirtualBox to Hyper-V

Recently I decided I wanted to install Docker Desktop for Windows in order to do some local testing.  The installation process is pretty straightforward but requires Hyper-V be enabled on the machine.  This is automatically done by the installer, but contains one minor inconvenient caveat: once Hyper-V is enabled, VirtualBox no longer functions.  My understanding for this is that Hyper-V is a ring-1 hypervisor which runs your primary OS as an (albeit special) VM.  Because of that, it's no longer possible to run a ring-2 hypervisor like VirtualBox.

There's plenty of guides out there on how to make this migration, but they all seem to be missing at least a few important points.  For the most part I followed this guide which has links to all of the required tools.  Here's the list of issues that I faced that were not covered in that article:

1. Convert-VHD would not recognize by vmdk file.  Ultimately I ended up finding this article which proposed an alternate command: ConvertTo-MvmcVirtualHardDisk. This worked as described in the original article.

2. The original article is a little vague on which lines might need to be commented out in the "desc.txt" file that is created.  I ultimately ended up needing to comment out these:

#ddb.uuid.image="8ce66459-16da-4a18-a56b-6149a49e0de3"
#ddb.uuid.parent="00000000-0000-0000-0000-000000000000"
#ddb.uuid.modification="00000000-0000-0000-0000-000000000000"
#ddb.uuid.parentmodification="00000000-0000-0000-0000-000000000000"

3. Once those issues were resolved, I had a new vhdx that I was able to attach to a new Hyper-V VM.  Unfortunately it wouldn't boot.  On a normal boot the VM would hang at the graphical boot screen for a few minutes, then fail and dump to the dracut recovery shell.  A look at journalctl indicated that this was because none of the disks were found.  A lot of articles on this issue recommend rebuilding the initramfs by rebooting and choosing the "rescue" option on the GRUB boot menu.  I did find that the rescue option would boot, but unfortunately rebuilding initramfs with: dracut --force didn't seem to have any effect.  Luckily the nice folks on freenode IRC #centos pointed me in the right direction:  my virtualbox VM had the guest additions installed still!  This article explains how to remove the vbox guest additions, but basically just run the installer again with the uninstall argument:

sudo sh /media/VBOXADDITIONS_4.1.10_76795/VBoxLinuxAdditions.run uninstall

Obviously hyper-v has no menu item to mount the guest additions CD, but the .iso can be found at:

C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso

Once that was uninstalled, one final reboot and... bob's your uncle!

Structure com.vmware.appliance.recovery.backup.job.details.info has a union with a field not required for this case = end_time

A VCSA 6.7 backup may fail with this error.  The most common explanation is that this is related to file permissions as described in this article.  But what if you're not using an IIS ftp server?  I was testing this using FileZilla FTP Server on Windows 10 1809.  In my case the service was running as SYSTEM, which already had full access to the FTP directory.  In fact it was creating subdirectories, but would fail when attempting to upload a file.

What I discovered was that the VCSA uses passive mode FTP with no option to change that fact.  By default passive mode servers choose random ports >1023 and pass them to the client which will then open a data connection to that port as described here. Unfortunately the Windows Firewall on this machine was set in the correct restrictive fashion, so those passive mode ports were never permitted.

It is possible, however, to choose custom passive mode ports in FileZilla.  In the server settings I chose to use the range 1024-1048, and then whitelisted those same ports in the firewall.  After that the backup completes successfully. 

Disable POP3 and IMAP on all mailboxes Office 365

I was recently annoyed to learn that there is no tenant wide way to disable POP3 or IMAP in Exchange Online.  Luckily PowerShell makes this task quite simple.  This article discusses the various commands that are available, but essentially you'll need to connect to Exchange Online via PowerShell and then run:

Get-User -ResultSize Unlimited | Set-CasMailbox -PopEnabled $false -ImapEnabled $false

Unfortunately you'll have to remember to disable these protocols for every new mailbox that is provisioned. This can also be performed from the ECP in the Mailbox Features.

Install vSphere 4 client on Windows 10

I ran into a number of errors attempting to install an older vSphere client in Windows 10. The first error says "This product can only be installed on Windows XP SP2 and above".  This issue is described in this VMware KB article.  I fixed this by right-clicking the installer executable and selecting to run it in Windows XP SP3 compatibility mode.  After that the installer seemed to run but would ultimately silently fail without installing anything.  It turns out that the installer requires .NET 3.5 to be installed.  These instructions from Microsoft show how to install this prerequisite in Windows 10, and it did not require a reboot for me.  Once vSphere 4 was installed I was able to successfully run it as well as update to later version when connecting to vCenter servers on newer ESXi.

Spiceworks HTTPS Redirect Breaks Stuff

We recently moved our Spiceworks installation to HTTPS.  While there's a handy setting in the options to force user connections to the portal to use HTTPS, this doesn't affect the backend used by admin and helpdesk staff.  There's a lot of bad advice out there about how to accomplish this redirect. Many threads like this one suggest adding a 302 redirect to a port 80 virtualhost to redirect to https.  While this does appear to work initially you will find that incoming emails no longer generate tickets.   If you view the production.txt log in C:\Program Files(x86)\Spiceworks\log\ you'll see an entry like this:

I[08:12:09.44 9b1030] scheduled call to check for ticket email url_ping: /tickets/check_email (http://127.0.0.1:80/tickets/check_email)
W[08:12:09.44 9b1030] check for ticket email url_ping: /tickets/check_email => unexpected response: Net::HTTPMovedPermanently

Yep, that's right - Spiceworks uses an internal API on port 80.  What's worse is that it does not follow the 302 redirect correctly, so if you go this route it will not work.  Luckily there are a number of other threads like this one on the Spiceworks forum that have a better suggestion - using Apache mod_rewrite to accomplish the task.  Specifically you will need to add the following directives to the httpd.conf file in C:\Program Files(x86)\Spiceworks\httpd\conf:

RewriteEngine On
RewriteCond %{REMOTE_HOST} !^127\.0\.0\.1
RewriteCond %{HTTPS} !=on
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]

This will also require that you have the directive:

LoadModule rewrite_module modules/mod_rewrite.so

somewhere in the config but this appears to happen by default.  Essentially this will rewrite any HTTP requests to HTTPS with the exception of 127.0.0.1, the loopback addressed used by Spiceworks for internal API calls.