"pandas.errors.EmptyDataError: No columns to parse from file" error from stable diffusion

 Stable Diffusion Error

I ran into this issue after the disk on which I was running stable diffusion AI image generator ran out of disk space.  0 bytes free.  Somehow the headings row of txt2img_logs.csv was removed, and the script wouldn't run.

Traceback (most recent call last):
  File "optimizedSD/optimized_txt2img.py", line 188, in <module>
    logger(vars(opt), log_csv = "logs/txt2img_logs.csv")
  File "E:\Projects\stable-diffusion\optimizedSD\optimUtils.py", line 57, in logger
    df = pd.read_csv(log_csv)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\util\_decorators.py", line 211, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\util\_decorators.py", line 317, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 950, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 605, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 1442, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\readers.py", line 1747, in _make_engine
    return mapping[engine](f, **self.options)
  File "C:\Users\User\.conda\envs\ldm\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 92, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas\_libs\parsers.pyx", line 554, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

The solution was to simply remove the log file which allowed the script to re-create it.

The headings are the following in case you need to recreate it for some reason:

prompt,    outdir,    skip_grid,    skip_save,    ddim_steps,    fixed_code,    ddim_eta,    n_iter,    H,    W,    C,    f,    n_samples,    n_rows,    scale,    device,    from_file,    seed,    unet_bs,    turbo,    precision,    format,    sampler,    ckpt

It's worth noting that I'm actually running OptimizedSD as it works well  (albeit slower) with less video ram.  There's some chance the log format is changed. 

WSUS.msc RSAT administration tool missing after Windows 10 upgrade

 After upgrading my laptop's version of Windows 10 form 1803 to 1809 recently I discovered that the WSUS administration tool from RSAT was missing.  The .msc file was actually still present but it wouldn't run.  

I considered downloading RSAT again, since that's how I had initially installed the various RSAT tools a few years ago, but found on this Microsoft page new instructions:

"Starting with Windows 10 October 2018 Update, RSAT is included as a set of "Features on Demand" right from Windows 10. Do not download an RSAT package from this page. Instead, just go to "Manage optional features" in Settings and click "Add a feature" to see the list of available RSAT tools. Select and install the specific RSAT tools you need. To see installation progress, click the Back button to view status on the "Manage optional features" page."

Seems easy enough, but it didn't work.  The install failed every time with the error message: "Install failed. Contact your administrator to get this feature."

 I searched for the error message and found this helpful blog post with the following instructions:

1. Disable WSUS by setting HKLM/Software/Policies/Microsoft/Windows/WindowsUpdate/AU/ to 0
2. Restart the Windows Update service
3. Install again

This article doesn't mention reverting the registry change, but I think that's probably warranted. Now it's back to mucking with WSUS!

UPDATE: This appears to affect other RSAT components as well.  The Group Policy Management Console gpmc.msc was also missing.  The same fix applied, so this isn't exclusive to the WSUS RSAT component.

 

Migrate a Centos 7 VM from VirtualBox to Hyper-V

Recently I decided I wanted to install Docker Desktop for Windows in order to do some local testing.  The installation process is pretty straightforward but requires Hyper-V be enabled on the machine.  This is automatically done by the installer, but contains one minor inconvenient caveat: once Hyper-V is enabled, VirtualBox no longer functions.  My understanding for this is that Hyper-V is a ring-1 hypervisor which runs your primary OS as an (albeit special) VM.  Because of that, it's no longer possible to run a ring-2 hypervisor like VirtualBox.

There's plenty of guides out there on how to make this migration, but they all seem to be missing at least a few important points.  For the most part I followed this guide which has links to all of the required tools.  Here's the list of issues that I faced that were not covered in that article:

1. Convert-VHD would not recognize by vmdk file.  Ultimately I ended up finding this article which proposed an alternate command: ConvertTo-MvmcVirtualHardDisk. This worked as described in the original article.

2. The original article is a little vague on which lines might need to be commented out in the "desc.txt" file that is created.  I ultimately ended up needing to comment out these:

#ddb.uuid.image="8ce66459-16da-4a18-a56b-6149a49e0de3"
#ddb.uuid.parent="00000000-0000-0000-0000-000000000000"
#ddb.uuid.modification="00000000-0000-0000-0000-000000000000"
#ddb.uuid.parentmodification="00000000-0000-0000-0000-000000000000"

3. Once those issues were resolved, I had a new vhdx that I was able to attach to a new Hyper-V VM.  Unfortunately it wouldn't boot.  On a normal boot the VM would hang at the graphical boot screen for a few minutes, then fail and dump to the dracut recovery shell.  A look at journalctl indicated that this was because none of the disks were found.  A lot of articles on this issue recommend rebuilding the initramfs by rebooting and choosing the "rescue" option on the GRUB boot menu.  I did find that the rescue option would boot, but unfortunately rebuilding initramfs with: dracut --force didn't seem to have any effect.  Luckily the nice folks on freenode IRC #centos pointed me in the right direction:  my virtualbox VM had the guest additions installed still!  This article explains how to remove the vbox guest additions, but basically just run the installer again with the uninstall argument:

sudo sh /media/VBOXADDITIONS_4.1.10_76795/VBoxLinuxAdditions.run uninstall

Obviously hyper-v has no menu item to mount the guest additions CD, but the .iso can be found at:

C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso

Once that was uninstalled, one final reboot and... bob's your uncle!

Structure com.vmware.appliance.recovery.backup.job.details.info has a union with a field not required for this case = end_time

A VCSA 6.7 backup may fail with this error.  The most common explanation is that this is related to file permissions as described in this article.  But what if you're not using an IIS ftp server?  I was testing this using FileZilla FTP Server on Windows 10 1809.  In my case the service was running as SYSTEM, which already had full access to the FTP directory.  In fact it was creating subdirectories, but would fail when attempting to upload a file.

What I discovered was that the VCSA uses passive mode FTP with no option to change that fact.  By default passive mode servers choose random ports >1023 and pass them to the client which will then open a data connection to that port as described here. Unfortunately the Windows Firewall on this machine was set in the correct restrictive fashion, so those passive mode ports were never permitted.

It is possible, however, to choose custom passive mode ports in FileZilla.  In the server settings I chose to use the range 1024-1048, and then whitelisted those same ports in the firewall.  After that the backup completes successfully. 

Disable POP3 and IMAP on all mailboxes Office 365

I was recently annoyed to learn that there is no tenant wide way to disable POP3 or IMAP in Exchange Online.  Luckily PowerShell makes this task quite simple.  This article discusses the various commands that are available, but essentially you'll need to connect to Exchange Online via PowerShell and then run:

Get-User -ResultSize Unlimited | Set-CasMailbox -PopEnabled $false -ImapEnabled $false

Unfortunately you'll have to remember to disable these protocols for every new mailbox that is provisioned. This can also be performed from the ECP in the Mailbox Features.

Install vSphere 4 client on Windows 10

I ran into a number of errors attempting to install an older vSphere client in Windows 10. The first error says "This product can only be installed on Windows XP SP2 and above".  This issue is described in this VMware KB article.  I fixed this by right-clicking the installer executable and selecting to run it in Windows XP SP3 compatibility mode.  After that the installer seemed to run but would ultimately silently fail without installing anything.  It turns out that the installer requires .NET 3.5 to be installed.  These instructions from Microsoft show how to install this prerequisite in Windows 10, and it did not require a reboot for me.  Once vSphere 4 was installed I was able to successfully run it as well as update to later version when connecting to vCenter servers on newer ESXi.

Spiceworks HTTPS Redirect Breaks Stuff

We recently moved our Spiceworks installation to HTTPS.  While there's a handy setting in the options to force user connections to the portal to use HTTPS, this doesn't affect the backend used by admin and helpdesk staff.  There's a lot of bad advice out there about how to accomplish this redirect. Many threads like this one suggest adding a 302 redirect to a port 80 virtualhost to redirect to https.  While this does appear to work initially you will find that incoming emails no longer generate tickets.   If you view the production.txt log in C:\Program Files(x86)\Spiceworks\log\ you'll see an entry like this:

I[08:12:09.44 9b1030] scheduled call to check for ticket email url_ping: /tickets/check_email (http://127.0.0.1:80/tickets/check_email)
W[08:12:09.44 9b1030] check for ticket email url_ping: /tickets/check_email => unexpected response: Net::HTTPMovedPermanently

Yep, that's right - Spiceworks uses an internal API on port 80.  What's worse is that it does not follow the 302 redirect correctly, so if you go this route it will not work.  Luckily there are a number of other threads like this one on the Spiceworks forum that have a better suggestion - using Apache mod_rewrite to accomplish the task.  Specifically you will need to add the following directives to the httpd.conf file in C:\Program Files(x86)\Spiceworks\httpd\conf:

RewriteEngine On
RewriteCond %{REMOTE_HOST} !^127\.0\.0\.1
RewriteCond %{HTTPS} !=on
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]

This will also require that you have the directive:

LoadModule rewrite_module modules/mod_rewrite.so

somewhere in the config but this appears to happen by default.  Essentially this will rewrite any HTTP requests to HTTPS with the exception of 127.0.0.1, the loopback addressed used by Spiceworks for internal API calls.

Hybrid Exchange Writeback Permissions

I recently ran into an issue after configuring Azure Active Directory Connect with hybrid Exchange that certain attributes couldn't be written back to the on-prem directory.  This manifests as errors in the sync tool, specifically a "Connected data source error code 8344" and "Insufficient access rights to perform the operation" on the export task. There's plenty of documentation that shows which permissions are required to support writeback of exactly 8 attributes.   For some reason it seems that the AAD Connect setup tool does not correctly add these permissions when selecting Hybrid Exchange mode.  There's a number of scripts out there, but two that I'll point out are this one from the Technet Gallery which appears to support a number of different configuration scenarios, as well as this one from c7solutions which is quite simple but effective.  The c7 post also has a great explanation of why these types of scripts are necessary.  The script is so useful that I've also generated an archive of the page here, in case it is ever moved/removed.

After running the script for a couple of minutes most export errors were resolved.  The specific issue can also be caused by an AD object with blocked inheritance.  This script from the technet gallery can be used to discover which users have inheritance blocked.  Once found they can either be fixed, or could be manually targeted for permissions with the aforementioned scripts.

Princeton Bitcoin Textbook

In case you hadn't heard Bitcoin hit at an all-time high of over $2,400 USD / BTC today.  There's plenty of good cursory information about Bitcoin but if you're looking for a decent in depth discussion of Bitcoin, related protocols, and other associated topics check out this book published by Princeton University Press.  The book is particularly suited to readers who already have an understanding of cryptography, computer science, networking etc.  If you're interested in going even deeper there's also a Coursera course to accompany it developed by one of the authors of the book.

O365 Migration Endpoint Creation Error

When creating a migration endpoint you may receive the error that "No MRSProxy was found running at 'name.domain.com'" with the name of your email server from autodiscover.  If you check the EWS virtual directory you will see that in fact the MRSProxy is enabled.  Further, if you check the application event log on the Exchange server you will see Event ID 1309 from Source ASP.NET.  This was a very frustrating error as it prevented the creation of migration endpoints on either the Exchange on-prem or online side of the equation.  Luckily I came across this thread which explained that it's necessary to  recycle the MSExchangeServicesAppPool on the on-prem Exchange server.  This was a quick fix with no observable impact to users.  After performing this step migration endpoint creation was quick and painless.

Audit File System in Server 2012 R2 and Event 4656

When I recently enabled file system auditing on a Windows Server 2012 R2 I was overwhelmed by the volume of events generated.  I'm talking >4 GB/day in some instances!  When I manually inspected the security event logs it appears to me that the majority of the events generated were event id 4656.  According to all of the documentation I can find this event should only be logged if the Audit Handle Manipulation subcategory of Object Access auditing is enabled.  In my case I hadn't enabled it!  Even the official MS documentation doesn't mention that event.

When searching for info I came across this comment at the bottom of a serverfault post:

Currently, under Server 2012 R2 events 4656 will generate even if Handle Manipulation category is disabled. In our case, we have enabled Audit File System category which was only generating 4660-4663 events on previous Server versions (2008-2008R2-2012) but on Server 2012 R2 this initiates overwhelming flow of 4656 events. The issue has been reported to Microsoft however there is no resolution yet.

I've yet to come across any official discussion of this, but  this certainly corresponds with my experience.  This thread was from June 2016, with a followup comment in January 2017.  Is this an undocumented feature?  A bug?

Luckily for me I'm using Splunk to ingest these logs so I was simply able to add:

blacklist1=EventCode="4656"

in the [WinEventLog://Security] stanza of C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_windows\local\inputs.conf in order to filter it out.  YMMV!

Set management IP on a VLAN on Dell Force10 S55 Switch Stack

While these switches do have dedicated management ethernet ports, it's often simpler and neater to set a management IP on the normal out of band / management VLAN that's already trunked on the uplink. For whatever reason the official Dell knowledge base article on the subject leaves out a crucial detail! It's necessary to set a default route before communication can commence. If you haven't set an IP on the management port you can't issue the management route command. Instead you need to issue the following:

ip route 0.0.0.0/0 192.168.0.1

to set a normal default gateway for the switch. Once that's done you're all set!

VMCA Intermediate CA

In vCenter 6 (and maybe some earlier versions) it's possible to configure the vCenter server to act as a subordinate CA to your existing PKI and issue all certs as part of a trusted chain. There's lots of instructions on the VMware knowledge base, but there are at least two critical errors in the CLI guide.

On this page that describes the process of compiling the VMCA Chained Cert that includes your root (and any other intermediate certs) it clearly shows the order as:

-----BEGIN CERTIFICATE-----
Certificate of VMCA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Certificate of intermediary CA
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Certificate of Root CA
-----END CERTIFICATE-----

But if you try this you will receive errors regarding an invalid cert chain. Specifically you will see:

Error Code : 70063
Error Message : Invalid Certificate Chain was gives as input

That's because this is exactly backwards! The new cert needs to be at the top, followed by the intermediate, then the root. I learned that on a couple of different blogs. I was hesitant to believe it at first, but once I found it in multiple sources I gave it a shot and sure enough it worked.

The second error relates to the vpxd service not restarting in time, leading the certificate-manager to attempt to rollback the changes (which fails). This situation is described in this forum posting, but no answer is given. After many hours of testing, and digging through logs, etc, a possible solution was discovered on this only tangentially related forum thread: nowhere in the documentation is there any mention of required OU entries for any of the certificates, but this blog post states that as of 6.0u1b and 6.0u2 there are in fact distinct requirements. They are as follows

For the MACHINE CSR use "Root" for Organizational Unit (OU)

For User Solution User Certificate CSRs:
For Machine, use "Machine" as OU
For vsphere-webclient, use "WebClient" as OU
For vpxd, use "VPXD" as OU
For vpxd-extension, use "VPXD-EXT" as OU

After changing these values the process ran through to completion.

My last headache was cause by my reading "stopping services" and "starting services" to mean that the services had been restarted. In fact it was necessary to restart all services (or the vcenter instance in my case) before all of the new certs took effect.

I hope this helps someone in the future avoid some of the pain I've experienced for the past two days.

X-IO Technology ISE Bad Password Immediately After Reset

We installed a rack of X-IO ISE 200, and 800 series SAN shelves for POC testing purposes this week. A random password was generated to replace the default and stored in our password safe as is our procedure. Unfortunately immediately after setting this password we could no longer log in with the new one, nor the default! A bit of hammering led me to discover that this was caused by the random password containing a backslash character. As soon as I removed the backslash and attempted to log in the password worked again. Apparently the set password routine stripped the backslash (probably sanitizing input) and set the password, while the login routine treated it as a valid password character. Could that mean that the authentication fields are not sanitized at all? I hope little Bobby Tables doesn't try to log in...

DIY Fiber Tester

Ever find yourself staring at a bundle of unknown fiber optic cables without your trusty (and expensive) Fiber Identifier? Fear not! As long as you're carrying your Android phone or iPhone you are in luck. Activate your LED flashlight and point one end directly into one strand of fiber. Check the fibers on the other end and you will see immediately which is which! I've uncovered a similar device on Instructables which would also be helpful but nothing beats the convenience of a device you're already carrying with you anyway.

Count all computer accounts in Active Directory

I needed to quickly get a count of how many computer objects:

Import-Module ActiveDirectory
Get-Adcomputer -Filter * | Measure-Object

gave me the info I was looking for!

Magic Buttons

Sudden mysterious spam increase with Exchange Online Protection

We've been using Exchange Online Protection and it's predecessor Forefront Online Protection for Exchange for about 5 years now. Somehow, suddenly, in the last couple of months we got a drastic increase in the amount of spam being delivered to users' mailboxes. I found, read, and even commented in this thread where other users were seemingly encountering the same issue.

Eventually I, with no support from MS, found the source of my issue though I'm not exactly sure how it came about.

We had always had our default content filter set to move spam and high confidence spam directly to the online quarantine. At some point this changed to what is now (and maybe always was) the default behavior of delivering the spam to the users' junk mail folder as described in this technet article. We did not have the necessary transport rules in place to actually cause the mail to go to junk email so it was just being delivered to the inbox.

I'm not clear if the default behavior changed, or if our rule had been modified and then subsequently reverted to default.

I can say that we signed up to demo Exchange Online so it's possible that reverted during the setup of our temporary "hybrid" demo situation. The timing seems about right but I can't say for certain this is the root cause of the issue.

High latency (ping) on every other packet

This morning I was troubleshooting some issues with out Aruba IAP105 access points (Dell branded). When attempting to ping either internal or external IPs I saw very strange behavior. Internally the first packet would have a latency of 1 or 2ms, the next over 900ms. This was perfectly regular and predictable, and external pings showed similar symptoms. Searching and searching and searching eventually yielded this link. Why yes, I was in fact running Metageek's inSSIDer tool in my troubleshooting attempts. Closing inSSIDer immediately fixed the high latency issue. I'm glad it only took me an hour of frantic searching to come up with this solution!

Install Filemaker Pro 7 on Windows 7

Filemaker Pro 7 came out in 2005, but one program in my agency was still using it for a particular database application. Upgrading would be expensive since all clients would need to be upgraded simultaneously as well as possible modifications needed on the database. Since we already own the software I went ahead and installed on a new client yesterday.

The first error is described in this forum post. After the installation was almost complete an error would pop up that openssl.exe had failed to run. Using the solution provided in this article seemed to work. I set the date back to 6/9/07 and didn't receive that particular error.

Frustratingly, though, the application would then fail to run with an error that read "This application has been installed incorrectly or modified by another program. Please run the installer to get a fresh copy of the application after determining the cause." None of the solutions provided by diligent googling seemed to apply (or work - I tried).

What did work was to start the installer in XP compatility mode. Instructions are shown here for using Windows XP compatibility mode. I didn't need to make any specific settings changes after install, and the OpenSSL error noted previously doesn't seem to have any actual affect on this process.