We upgraded NCM to 7.3.2 today and we're no longer receiving e-mail updates when changes are made to any of our devices.  I took a look at the most recent admin guide and updated the program that gets executed under Syslog Viewer from:

"C:\Program Files (x86)\SolarWinds\Orion\SolarWinds.NCM.RTNForwarder.exe" ${IP}

to:

C:\Program Files (x86)\SolarWinds\Orion\SolarWinds.NCM.RTNForwarder.exe ${IP_Address},RealtimeNotification,${DateTime},${Message}

but no luck.  Not really sure where to go from here.  Any thoughts?

I am trying to clean up my switch configurations. There are a lot of old outdated ntp servers and snmp trap servers and other stale info with in the configuration. I have been trying to find a way to remove all of these old server then run a script to only have the correct ones. Any ideas or do I need to go through each device and remove them individually ? I wouldnt mind if the new server was list and got removed.

Example would be:

ntp server 1.1.1.1

ntp server 1.2.2.2

ntp server 3.3.3.3

runs the following

no ntp server 1.1.1.1

no ntp server 1.2.2.2

no ntp server 3.3.3.3    --- This is fine if it does this.

ntp server 3.3.3.3

Ok, fairly simple compliance rule.  Checking all tunnel interfaces to make sure they have CDP enabled.   Everything works fine, the remediation script is fairly simple, just:

${ConfigBlockStartLine}

cdp enable

But when I hit "Execute Script" it appears to do nothing, it just sits there.  So, I wait a while and then hit "Execute Script" again, still nothing.

I finally tried opening the "Transfer Status" screen, I see some of these jobs running, I can confirm it with the "show upload results".  I've done it a few times, and it appears to execute on some of the devices, initially it found around 420.   After updating the config cache, it had around 350.    The next time it got down to 320.  So its executing some of the jobs, but not all.

This time around, I hit "execute script" about 5 minutes ago, still looks like nothing is happening.   So far it seems as if its generated around 50+ jobs?   Can't tell for sure because RTCD jobs are getting spun off as this one works...   I am having problems clearing the complete jobs too, which is odd.   The CPU on the Orion box is at maybe 10-15%

I've tried cycling all the services just before doing it too, just to make sure nothing was causing problems.  Same result.

I think I'm running the latest versions...

Orion Platform 2015.1.2, IVIM 2.1.0, QoE 2.0, OFSMM 6.6, UDT 3.2.2, NCM 7.4, NPM 11.5.2, IPAM 4.3, NTA 4.1.1

Anyone else having this problem or anything like it?    Will probably open a ticket in a bit...

I've been setting up a bunch of compliance reporting as of late and, while I now have a rule that looks for the correct logging servers, I would also like to have one that looks for any other logging servers that might be setup.

I've been banging away at it for a while now, but I just don't seem to have found a filter that doesn't either fail everything or pass everything...

Here's what I've got at the moment:

------------------------------------

Match if String is Found

Must NOT Contain String: logging host x.x.x.x

Must NOT Contain String: logging host y.y.y.y

Must NOT Contain String: logging host z.z.z.z

Must Contain String: logging host

All four are "AND" with no use of parenthesis.

------------------------------------

This ALWAYS comes through as a "Pass," even if there is a fourth logging host that is NOT one of those other three.  I have tried a variety of regex variations, but it seems to invariably pass or fail 100% of the devices.  It is totally possible that I got close with the regex, but just didn't know enough.

In the case of the current, simplified string-only attempt, is is possible that it is excluding the first three strings, then going right back and matching the fourth filter against those same three lines?  In my head, it should only cause a match if a logging host is in the config and does NOT match one of the three known IPs.

Thanks for any advice!

Hello

Im fairly new to solarwinds RTCD so excuse me if this is a simple fix but we have successfully configured our Cisco ASA's and Catalyst 3850 switches to send their RTCD updates to our Orion Server. I am not having issues with  triggering the alert when when a change is made, but instead I am experiencing an issue with the email formating. I am using the HTML format option that can be found the in NCM RTCD settings as well as the RTCD logging. When we make a change to a device the email notification is saying the change was made by a different user than the one shown in the syslog message.  The username we get is shown as " RealtimeNotification" , this is not a username that we use to log into our switches and it is confusing why it is appearing on these emails when the syslog messages have the correct user names being recorded.

We currently are using the standard  NCM change detection rule for our Catalyst 3850 switches which is functional. For the ASA's we are using a rule based on the syslog messages ASA-5-111008 & ASA-5-111010. if the message is found it will

then execute the following external program:

C:\Program Files (x86)\SolarWinds\Orion\SolarWinds.NCM.RTNForwarder.exe ${IP_Address},RealtimeNotification,${DateTime},${Message}

I have checked several kb's and  thwack articles and looked into how RTCD gets the username information from the syslog messages but came up empty handed. I believe it maybe an issue with how the syslog message data is carried over/included in ${Message} Macro at the end of the execute external program action for the alert action but that is a hypothesis.

Has anyone else experienced this issue and found a resoultion?

Can any confirm this information is correct or shed some light to how RTCD accomplishes including the username information in the email notifications?

Any help would be beneficial.

Thanks you!

I was wondering if there were any issues that arose during the upgrades to these versions?

We recently loaded NCM 7.4 with hopes of using the tool to help automate reporting of potential security vulnerabilities in our Cisco IOS devices.

While the framework in NCM looks great, the source of data (CVEs) is lacking - in that it appears to only look at major releases, i.e...  12.2, 12.4, but does not take into account the minor release info, i.e... 12.4(24)T4.

What I end up with is a report of my 300 devices, with hundreds of "potential" vulnerabilities - even though the IOS is a current release.

What I am curious is if Solarwinds will be enhancing the NCM product in order to obtain more detailed vulnerability reporting capabilities through the use of CVRF files.

Thanks in advance for your response!

Hello,

We use SolarWinds to monitor both servers and network devices, but only use NCM to manage configs for network gear. We started running discoveries to make sure we didn't miss any devices that should be in SolarWinds for monitoring. The problem we are running into is that if you actually discover new nodes and want to import them, they seem to get imported into NCM automatically. I don't see a setting to not manage them with NCM. So after almost every import we run out of NCM licenses and have to manually remove these newly discovered nodes from NCM.

Is there a setting for the discover that I'm missing somewhere? Our discoveries mostly find servers and I don't want them to eat up our NCM licenses. And it's a pain to remove them manually from NCM every time an import is done.

Thanks in advance for your help!

NCM 7.x has several node management improvements (common with Orion Core/NPM, new Discovery Sonar…).
Due to these changes, NCM 7.x has stricter rules related to node uniqueness and duplicate nodes are not allowed anymore in NCM 7.0 (because NCM nodes are being handled by Orion Core, which does not support duplicate nodes).

This NCM 6.x capability was sometimes used to deal with devices such as the Cisco ASA, which require management of multiple contexts.

The solution in NCM 7.x (can work in NCM 6.1 as well) uses multiple Config Types:

Solution:

1.       In Win32 NCM application, add a new Config Type for each context (for example Context1, Context2).

2.       Specify in your device template, how to switch to each context (see example below).

3.       Include the${ConfigType} macro in the Reset Command.

4.       Make sure to specify the RegEx value in the Reset command. Depending on your prompt settings, the value should be '#', '>', ']', etc.

5.       Avoid using ${ConfigType} macro in the DownloadConfig command. Make sure the command is the right one, e.g. "show running-config".

Device Template Example:

<Command Name="RESET" Value="${ConfigType}" RegEx="#" />

<Command Name="Context1" Value="command to switch to Context1"  />

<Command Name="Context2" Value="command to switch to Context2 "/>

<Command Name="DownloadConfig" Value="show running-config"/>

Explanation:

When NCM downloads a configuration, the first command issued is the ‘Reset’ command defined in the device template.

The ${ConfigType} macros will be replaced by the appropriate switch context command, based on the config type selected for the dowload.

The context will be dynamically switched before the download command is executed.

When the context is changed, so is the prompt. In order to recognize the prompt, a regular expression (RegEx) must be used to ignore the variable part of the prompt.

Example:

Let’s assume the command mode prompt after login is : Tex-3750#

-          After switching to Context1, the command mode prompt becomes Tex-3750-Context1#.

-          After switching to Context2,the command mode prompt becomes Tex-3750-Context2#.

The RegEx to use in the reset command should match the common portion of the prompt in both contexts. In this example this could be RegEx="#".

An NCM user shared the result of his work (FWSM on NCM 6.1):

Should work the same in NCM 7.0. Thanks csowerby

Cisco FWSM Virtual Context Template

Message was edited by: Jiri Cvachovec

Hello,

 I was looking through our NCM web interface and saw the startup vs running config pie chart.  I noticed that there were a small portion of the devices that did not match so I started looking into them.  About half of them matched but were still reported as not matching.  I checked both through the web interface and through the console and there were no differences.  Has anyone else seen this?

 I know for sure there were a couple changes that I made to the devices that I did a "write mem" after I did them and one of those devices showed up on the list.

 Thanks!

we have network devices dsitributed across 4 pollers ( 1 primary and 3 additional). we are running a NCM nightly backup config job daily to download the running config from devicea dn job completion emails are being received from just 2 pollers and we are not getting emails for the other 2 pollers including primary.

could anyone help me in resolving this issue. I ahve alreday restarted the jon engine services on both the pollers for which I am not gettting the results from. however when I got to the pollers to NCM to see the running config status I see backup is taking place but I am not receiving the status of those 2 pollers.

Please hlp me in resolving the issue.

Thanks

Akansha

I frequently used the server-based NCM's 7.3 (and earlier) application to display the last time data was input on Cisco ports.

I used the information from this field whenever a Desktop Support Tech would call to say a switch was out of open ports.  It was a quick and easy matter to open NCM's app on the server, select the switch and pull down the Cisco Last input option.  Voila!  The last time all ports received incoming data  was present in an exportable and sortable column.

If there were ports that had been down for a very long time, I'd have the tech unpatch them for reuse.  If there were none, it was time to order another switch.

NCM 7.4 doesn't allow running the application on the server--only via the Web UI.

The 7.4 Web UI has a similar looking drop down to that of the  NCM 7.3 on-board-server-app, but the Last Time Data Was Transmitted or Received option / column is not present.

A brief conversation with Jiri at SW suggested this may have been removed and repurposed for UDT.

Can anyone show Cisco interface "Last input" info in NCM 7.4 today?  Am I missing an option in getting to to show in NCM 7.4?

The Last input option is not present to select as a column.

A CLI session to the same switch shows the information present in a "show interface" command:

  GigabitEthernet1/46 is up, line protocol is up (connected)

  Last input 5w6d, output never, output hang never

Is this functionality really gone from NCM 7.4?

Can I manually recover it with a Universal Device Poller?   But that wouldn't work for every Cisco device, would it, since there are so many different models of switches?

I downloaded the free version of IP Address tracker last night and it was close to 700MB.  It installed IIS and SQL Server 2008 R2 Express and whole bunch of other stuff.  Is the old IP Address tracker no longer available?

This is what I see when I try to download it. Am I missing something?

Hello, I have NCM 7.2.2 with 4 polling engines and currently rely on the fact that only the main polling engine is the only one who performs the configuration activities due to firewall and compliance reasons.  I understand that after 7.3, this is no longer an option and the NCM functions are distributed to each polling engine. 

My question is this:  is there a way to keep the main polling engine as the master for all NCM functions but yet be assigned to a different polling engine after version 7.3?

Anyone else run into this?  For me this means a total redesign of our environment.

Thanks!

Hi,

I'm trying to exclude line with date of my comparison criteria but for now it doesn't seems to works.

my configs looks like :

#

# Tue Sep 08 00:20:45 2015 CEST

# box type             : VSP-8284XSQ

# software version     : 4.2.1.0

# cli mode             : ACLI

And I want to exclude the 2nd line

I've tried this regex : ^# (Mon|Tue|Wed|Thu|Fri|Sat|Sun)

I also tried : ^(# )(Mon|Tue|Wed|Thu|Fri|Sat|Sun)

what did I miss ?

Hi,

I have got some issues when I am trying to update my switch firmware through SCP/SFTP Server which is integrated on Solarwinds Network configuration Manager.

SSH is allowed between Solarwinds Server and Cisco devices. If you faced the same problem please guide me how can we solve this issue ?

I have enabled the SCP server and created one <user > without any password.

Copied the IOS image on X:/sftproot - Solarwinds Server.

Start SFTP server on Solarwinds Server.

Execute Command on switch  :

Router1#copy scp: flash:
Address or name of remote host []? 10.x.x.x
Source username [user1]? user
Source filename []? image_filename.bin
Destination filename [image_filename.bin]?

%Error opening scp://user@10.x.x.x/image_filename.bin (Undefined error)
Router1#

I'm currently having an issue getting NCM RTCD to work all the time. In previous jobs I had RTCD set to check 10 min after anyone logged on. I had no trust in my techs, now I trust them and no longer need to be big brother and just want to know if a command gets lost after a reboot.

We are a Cisco shop and have RTCD setup to watch syslogs for reboot messages like *System restarted*, *Reload*, *cold start* ,*rebooted*. The issue I'm having is sometime the change detection runs when a node is rebooted, some times it doesn't. In the web console we will see connection refused messages. When talking with support it was noted that NCM was trying to fast to connect to the device after the reboot was detected and was unable to log on as the node wasn't fully booted.

I have tried numerious different ways of creating a new alert in NPM to do the change dection, but have been unable to get it to run.

I curious as to if anyone else is doing RTCD on a reboot and having success at getting to work 100% of the time.

Thanks

I'm going through and updating our network devices. I'm faced with the option of configured SNMP traps or Syslog messages to relay information. If I have the option to use either, which one would be the better choice and why? The syslog messages, at least from the viewer perspective seem to be smaller and more concise. But I'm not sure of the pros and cons of each system.

My test Solarwinds Network Configuration Manager require to have a natted Address to log on to the end Network device.

i.e address of servers is 10.10.10.1

But to access the network device it must use the IP Address of 20.20.20.1

The Natted  IP Address of the server is 20.20.20.100. Any replys back to the server from the switch  needs to come back from this IP Address, or any SSH Access to the switch

I can't find the option can anyone help

Per direction of our IA department we recently updated the self-signed certificate on our SolarWinds Application Server (Windows Server 2008 R2 Enterprise platform) from 1024 bit to 2048 bit.  Here are the instructions we followed: https://thwack.solarwinds.com/community/solarwinds-community/geek-speak_tht/blog/2012/10/23/getting-certificates-up-to-speed--updating-rsa-key-security

We carefully followed the process, but now our Network Configuration Manager is broken:

1.  It will not download or upload Cisco startup or running configurations.  The error message we receive is: "Start Transfer Error.  See NcmBusinessLayerPlugin log for details" "Fix connection in Device Template."  When we click on the "Fix connection" link in the Configuration Manager/Transfer Status tab, then navigate to the General Device Access tab, and we verify the settings and press "Test," we receive the error: "Unable to connect to polling engine (server name) on the relevant server.  Verify that NCM 7.4 or later is installed on the server."  We are running NCM 7.4.

2.  We cannot edit nodes, even if we delete them and re-add them.  When we attempt to edit a node, we receive the following message: "There was an error retrieving data from SolarWinds Information Service" and "Invoke failed, check fault information."

3.  We have several WMI (Windows) credentials stored.  They can be verified in Settings/Windows Credentials/Manage Windows Credentials.  However, when we add a new Windows Server 2008 R2 Enterprise node and select the WMI option from the Windows Servers: WMI and ICMP/Choose credential/<New Credential> arrow, no options are available.  If we attempt to type the stored credential name manually, it still does not appear.

4.  We CAN run default SolarWinds reports from the CONFIGS tab/Reports menu option in the web interface.  However, none of our scheduled jobs will run.  Everything (including the items listed above) was working fine until the certificate update.

These behaviors persist regardless of browser (Firefox 41.0.2 or Internet Explorer 10.0.32)

We opened a support case with SolarWinds three weeks ago and have been communicating with them daily via telephone and email.  They have walked us through an array of troubleshooting efforts including registry fixes, but so far nothing is working.

The main log files we have been dealing with in our troubleshooting efforts are:

- BusinessLayerHost.log

- Core.BusinessLayer.log

- InformationService.log

- NcmBusinessLayerPlugin.log

- Orion.InformationService.log

- OrionPermissionChecker.log

- OrionWeb.log

The licensed products we are running are:

- Orion Platform 2015.1.2

- NCM 7.4 (NCM-NPM Integration 7.4)

- SAM 6.2.2

- NPM 11.5.2

- IPAM 4.3

- NTA 4.1.1

Also:

- DPA 10.0.0

- PM 2.1

And:

- SolarWinds Collector v2.12.38

- SolarWinds Job Engine v2.10.0

- SolarWinds Integrated Virtual Infrastructure Monitor v2.1.0

- SolarWinds Information Service v2015.1.6134

The Hot Fixes are also installed.

We are running Windows.NET Framework 4.5 and WinPcap 4.1.3.

Our SolarWinds Application Server and related servers (SQL and NTA) are located on a closed network with no direct access to the Internet.

The reason I've listed so many items is because the SolarWinds technical support and development teams have had us digging through all of them, weeding through logs, reports, program files and registry keys, and also repairing and uninstalling/re-installing nearly everything but to no avail.  We are close to making a decision to completely tear down the server and rebuild it from bare-bones scratch as if it were new hardware fresh out of the box.  We would really love to avoid that level of effort.

And no, we don't have the luxury of a test environment for this sort of thing, else we could have avoided this altogether in our production environment.

Has anyone else encountered this issue?  If so, what was your solution?

Many thanks for your patience in reading and considering this problem.